Jul 06 23:48:52.942032 ip-10-0-132-169 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.353947 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358409 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358422 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358425 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358429 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358432 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358435 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358438 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358441 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.366223 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358443 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358446 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358449 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358452 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358454 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358457 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358459 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358470 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358473 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358476 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358478 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358481 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358484 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358487 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358490 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358492 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358495 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358497 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358500 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358502 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.367234 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358505 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358508 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358510 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358513 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358515 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358518 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358520 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358523 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358526 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358530 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358535 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358539 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358542 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358545 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358548 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358550 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358553 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358557 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358560 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.368030 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358562 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358566 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358568 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358571 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358573 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358577 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358580 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358582 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358585 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358587 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358590 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358593 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358596 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358598 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358601 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358604 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358607 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358609 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358612 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.368613 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358615 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358617 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358620 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358623 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358625 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358628 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358631 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358633 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358636 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358638 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358641 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358644 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358647 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358650 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358652 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358656 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358659 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358663 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358666 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.369147 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358669 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358672 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358675 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358678 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358681 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358683 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358686 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358689 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358691 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358694 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358697 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358699 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358702 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.358704 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359473 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359480 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359483 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359486 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359490 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.369684 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359494 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359497 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359500 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359504 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359507 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359510 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359513 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359516 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359518 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359521 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359524 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359527 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359529 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359533 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359537 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359540 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359544 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359547 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359550 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.370196 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359553 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359556 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359559 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359561 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359564 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359566 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359569 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359571 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359574 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359577 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359579 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359582 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359584 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359587 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359590 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359593 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359596 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359599 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359601 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359603 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.370800 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359606 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359609 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359611 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359614 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359616 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359618 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359621 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359624 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359627 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359630 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359632 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359635 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359638 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359640 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359642 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359645 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359647 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359650 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359652 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359655 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.371321 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359657 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359660 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359662 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359666 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359669 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359671 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359674 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359678 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359681 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359683 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359686 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359688 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359691 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359696 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359699 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359702 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359705 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359708 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359711 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359714 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.371911 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359717 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359719 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359722 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359725 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359728 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359731 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359734 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359737 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359739 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359742 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359768 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359771 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359774 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359776 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.359779 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360183 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360189 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360192 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360196 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.372449 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360201 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360206 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360210 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360213 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360216 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360219 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360222 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360225 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360229 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360232 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360235 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360238 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360241 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360243 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360246 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360248 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360251 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360254 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360256 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.373008 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360259 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360262 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360264 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360267 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360269 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360272 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360274 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360277 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360279 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360282 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360284 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360287 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360289 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360292 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360298 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360301 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360304 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360306 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360309 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360311 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.373522 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360316 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360318 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360321 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360324 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360326 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360329 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360331 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360334 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360338 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360341 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360344 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360348 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360350 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360353 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360356 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360359 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360361 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360364 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360367 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.374113 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360369 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360372 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360375 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360377 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360380 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360382 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360385 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360389 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360392 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360394 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360397 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360400 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360402 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360406 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360409 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360411 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360414 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360416 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360419 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360421 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.374615 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360425 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360427 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360431 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360435 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360437 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360440 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360442 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360445 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360448 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360451 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360453 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360456 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360458 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360461 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360463 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360466 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.360468 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360544 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360552 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360561 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360565 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:48:53.375199 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360570 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360573 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360578 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360582 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360586 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360589 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360593 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360597 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360600 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360603 2556 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360606 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360609 2556 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360612 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360615 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360620 2556 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360623 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360626 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360629 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360633 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360636 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360639 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360642 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360646 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360649 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:48:53.376180 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360652 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360655 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360658 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360661 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360666 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360669 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360673 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360676 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360679 2556 flags.go:64] FLAG: --enable-server="true" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360683 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360687 2556 flags.go:64] FLAG: --event-burst="100" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360690 2556 flags.go:64] FLAG: --event-qps="50" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360694 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360697 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360700 2556 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360705 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360708 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360711 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360714 2556 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360717 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360721 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360724 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360727 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360729 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360732 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:48:53.376934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360735 2556 flags.go:64] FLAG: --feature-gates="" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360739 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360743 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360760 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360763 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360766 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360770 2556 flags.go:64] FLAG: --help="false" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360773 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360776 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360779 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360782 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360786 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360789 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360794 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360796 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360799 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360803 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360806 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360810 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360814 2556 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360817 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360820 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360823 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360826 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:48:53.494145 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360828 2556 flags.go:64] FLAG: --lock-file="" Jul 06 23:48:53.431284 ip-10-0-132-169 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360831 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360835 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360838 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360843 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360846 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360849 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360852 2556 flags.go:64] FLAG: --logging-format="text" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360855 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360858 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360861 2556 flags.go:64] FLAG: --manifest-url="" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360864 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360868 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360871 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360875 2556 flags.go:64] FLAG: --max-pods="110" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360878 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360881 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360884 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360887 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360889 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360892 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360897 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360904 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360907 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360911 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:48:53.495090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360914 2556 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360918 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360924 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360927 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360931 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360934 2556 flags.go:64] FLAG: --port="10250" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360937 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360940 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06ba10b12e6d775e9" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360943 2556 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360946 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360949 2556 flags.go:64] FLAG: --register-node="true" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360952 2556 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360955 2556 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360959 2556 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360962 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360964 2556 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360968 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360971 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360975 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360977 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360980 2556 flags.go:64] FLAG: --runonce="false" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360983 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360986 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360990 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360992 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:48:53.496066 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360995 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.360998 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361002 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361005 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361010 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361013 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361016 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361019 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361022 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361027 2556 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361030 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361036 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361039 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361042 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361046 2556 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361049 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361052 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361055 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361059 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361062 2556 flags.go:64] FLAG: --v="2" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361066 2556 flags.go:64] FLAG: --version="false" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361070 2556 flags.go:64] FLAG: --vmodule="" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361073 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.361077 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361209 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.497090 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361213 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361217 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361220 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361222 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361225 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361228 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361230 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361233 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361236 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361238 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361241 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361245 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361248 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361250 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361253 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361256 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361260 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361263 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361267 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361270 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.497854 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361273 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361276 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361279 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361282 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361284 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361287 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361296 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361301 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361304 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361306 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361309 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361312 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361314 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361317 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361319 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361322 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361325 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361327 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361330 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.498510 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361332 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361335 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361338 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361340 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361345 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361348 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361351 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361353 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361356 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361361 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361364 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361367 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361370 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361373 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361376 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361378 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361381 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361383 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361386 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.499117 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361389 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361391 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361394 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361397 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361399 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361402 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361404 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361408 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361412 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361414 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361417 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361420 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361422 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361425 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361428 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361430 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361433 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361437 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361439 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.499726 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361442 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361444 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361447 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361451 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361454 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361456 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361460 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361464 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361467 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361470 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361473 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361476 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361479 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361482 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361485 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361487 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361490 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361493 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361496 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361498 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.500285 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.361501 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.362115 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.370451 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.370468 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.371386 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.371394 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371454 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371461 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371465 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371468 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371471 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371474 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371477 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371480 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.500967 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371482 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371485 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371488 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371491 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371493 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371497 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371499 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371502 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371504 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371507 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371510 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371513 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371516 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371519 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371522 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371525 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371527 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371531 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371535 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.501412 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371538 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371541 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371545 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371547 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371550 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371555 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371558 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371562 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371565 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371567 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371570 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371573 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371575 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371578 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371581 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371584 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371587 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371589 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371592 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371595 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.502007 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371597 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371600 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371603 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371605 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371608 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371610 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371613 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371616 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371618 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371622 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371625 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371627 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371630 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371633 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371637 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371640 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371643 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371646 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371648 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371651 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.502559 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371654 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371657 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371660 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371663 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371665 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371668 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371671 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371673 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371676 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371679 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371681 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371684 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371687 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371690 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371692 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371695 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371698 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371700 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371703 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371706 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.503167 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371708 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371711 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371714 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371716 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371719 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371721 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371724 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371726 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371730 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371732 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371735 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371738 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.371753 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371867 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371873 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:53.503699 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371877 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371880 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371882 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371885 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371890 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371894 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371897 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371901 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371903 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371906 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371909 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371911 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371914 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371917 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371920 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371922 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371925 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371928 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371930 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:53.504164 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371934 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371938 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371940 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371944 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371946 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371949 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371952 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371955 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371958 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371961 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371963 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371966 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371969 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371972 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371974 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371977 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371980 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371983 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371985 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371988 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371991 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:53.504718 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371994 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371996 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.371999 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372001 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372004 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372006 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372009 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372011 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372014 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372019 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372021 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372024 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372027 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372029 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372032 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372035 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372038 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372040 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372043 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372046 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:53.505375 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372049 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372052 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372054 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372057 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372059 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372062 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372065 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372067 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372070 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372072 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372075 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372078 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372080 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372083 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372085 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372088 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372090 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372093 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372096 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372098 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:53.506102 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372101 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372105 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372107 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372110 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372112 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372115 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372118 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372121 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372124 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372126 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372129 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372131 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372138 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372141 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372144 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372146 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:53.506994 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:53.372149 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.372153 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.372266 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.374329 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.375721 2556 server.go:1019] "Starting client certificate rotation" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.375777 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.375858 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.380498 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.385418 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.395545 2556 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.400716 2556 log.go:25] "Validated CRI v1 image API" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.401428 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:48:53.707078 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.403895 2556 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bed8fdef-06a5-4450-8c14-edfbabdcbc62:/dev/nvme0n1p4 c4a28ae8-b60a-4ae4-b272-b43019596bc6:/dev/nvme0n1p3] Jul 06 23:48:53.725917 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.403913 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.409444 2556 manager.go:216] Machine: {Timestamp:2026-07-06 23:48:53.408184715 +0000 UTC m=+0.338242213 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096231 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e56f01359e3a222f1b80ac8cafa3a SystemUUID:ec2e56f0-1359-e3a2-22f1-b80ac8cafa3a BootID:4547dc9f-e481-4b5d-a15a-2660ca6b484e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:43:fe:da:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:43:fe:da:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:c5:4f:f5:25:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.409543 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.409627 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.412675 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.412711 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-169.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.413429 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.413439 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:48:53.726922 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.413447 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.414178 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.414260 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.415514 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418302 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418427 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418454 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418466 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418484 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418497 2556 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418512 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418893 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.418909 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.421001 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.424879 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.424909 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426062 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426077 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426083 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426089 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426097 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426103 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426482 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426488 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426507 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426514 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426522 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.426529 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.427268 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.427276 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.428878 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-169.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.428923 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.428991 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.430514 2556 server.go:1268] "Started kubelet" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.430911 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.432638 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.433498 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.433533 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.435066 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:48:53.727363 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.440305 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.440618 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.441695 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.441793 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxpmn" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.442317 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.442324 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.442337 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.442427 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.442438 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.446376 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxpmn" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449498 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449511 2556 factory.go:55] Registering systemd factory Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449520 2556 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449792 2556 factory.go:153] Registering CRI-O factory Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449806 2556 factory.go:223] Registration of the crio container factory successfully Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449843 2556 factory.go:103] Registering Raw factory Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.449858 2556 manager.go:1196] Started watching for new ooms in manager Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.450262 2556 manager.go:349] Starting recovery of all containers Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.451047 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.452080 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.454716 2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-169.ec2.internal\" not found" node="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.465788 2556 manager.go:354] Recovery completed Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.470936 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.472962 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.472987 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.472999 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.473503 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.473516 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.473536 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.474689 2556 policy_none.go:49] "None policy: Start" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.474703 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.474715 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.475705 2556 policy_none.go:47] "Start" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.476404 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.477257 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.477275 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.477285 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.526384 2556 manager.go:337] "Starting Device Plugin manager" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.526419 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.526429 2556 server.go:87] "Starting device plugin registration server" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.526743 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.526888 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.526998 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:48:53.728586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.527092 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.527101 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.527677 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.527732 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.581777 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.582937 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.582970 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.583010 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.583047 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.585010 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.627112 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.627968 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.628002 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.628013 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.628040 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.636884 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.637044 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-169.ec2.internal\": node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.652658 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.684663 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"] Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.684737 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.685680 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.685703 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.685713 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.687335 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.687491 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.687526 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688169 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688191 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688201 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688569 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688604 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.688616 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.689361 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.689386 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:53.729949 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.690082 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:53.730986 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.690110 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:53.730986 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.690120 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:53.730986 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.708427 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-169.ec2.internal\" not found" node="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.730986 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.712358 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-169.ec2.internal\" not found" node="ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.743666 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.743691 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.743709 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12306cecd9bb065dc086ef079ccc1976-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"12306cecd9bb065dc086ef079ccc1976\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.753452 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.844887 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.844919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.844938 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12306cecd9bb065dc086ef079ccc1976-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"12306cecd9bb065dc086ef079ccc1976\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.844981 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.844992 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b3a72da0cd12d86073bbd691ce262f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal\" (UID: \"1b3a72da0cd12d86073bbd691ce262f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:53.845043 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12306cecd9bb065dc086ef079ccc1976-config\") pod \"kube-apiserver-proxy-ip-10-0-132-169.ec2.internal\" (UID: \"12306cecd9bb065dc086ef079ccc1976\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.853601 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:53.993571 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:53.953776 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:54.239694 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.015655 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:54.239694 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.015814 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:54.239694 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.053875 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:54.239694 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.154510 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-169.ec2.internal\" not found" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.245134 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.342447 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.354552 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.356379 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.367169 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.375743 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.375876 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.375919 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.375929 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.419084 2556 apiserver.go:52] "Watching apiserver" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.441622 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.443190 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:54.445001 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.443923 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-tpbff","openshift-image-registry/node-ca-h6nbc","openshift-multus/multus-additional-cni-plugins-cj74h","openshift-network-operator/iptables-alerter-6bw7q","kube-system/global-pull-secret-syncer-ckczt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx","openshift-cluster-node-tuning-operator/tuned-2wg8h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal","openshift-multus/multus-vhl7b","openshift-multus/network-metrics-daemon-7xjzp","openshift-network-diagnostics/network-check-target-hhnlw","openshift-ovn-kubernetes/ovnkube-node-4b9rn","kube-system/konnectivity-agent-h9g4p","kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal"] Jul 06 23:48:54.449443 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.447508 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.449443 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.448062 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:43:54 +0000 UTC" deadline="2028-02-25 11:05:14.122591807 +0000 UTC" Jul 06 23:48:54.449443 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.448106 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14363h16m19.674487953s" Jul 06 23:48:54.449443 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.448464 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.450222 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.451424 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.451813 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.451911 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.451914 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zrwzl\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.452606 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.452661 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453022 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-29nl9\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453027 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453049 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453101 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453120 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453205 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453270 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vn7cp\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453405 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453467 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453475 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.453897 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.454875 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wxpdt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.454884 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.454903 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.454886 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.455214 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.456274 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.456385 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.456428 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.456658 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.456823 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.457065 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dwzqw\"" Jul 06 23:48:54.618736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.457802 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.457889 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.458083 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.458180 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.458247 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wdjr5\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.459736 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.459821 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.461685 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.461898 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9bl6t\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.463727 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.467029 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.467056 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9ngsl\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.467119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.469757 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.473139 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9gc2j\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474437 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474445 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474459 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474445 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474449 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.474568 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.487397 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qh47r" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.498092 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qh47r" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.545706 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-etc-kubernetes\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.620680 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-var-lib-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549060 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-bin\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-host\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549092 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549133 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-log-socket\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549202 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dbj\" (UniqueName: \"kubernetes.io/projected/21436535-c81e-4688-bc63-5c0720cba1cd-kube-api-access-b8dbj\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cni-binary-copy\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549265 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gpwk\" (UniqueName: \"kubernetes.io/projected/17331bcb-03ec-4c32-9fdd-d479965632fb-kube-api-access-5gpwk\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549288 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-kubernetes\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549304 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-tmp\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549344 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-system-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-netd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.621735 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549397 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17331bcb-03ec-4c32-9fdd-d479965632fb-iptables-alerter-script\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549463 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-modprobe-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549514 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-host\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549543 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-netns\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549581 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-multus\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ntx6\" (UniqueName: \"kubernetes.io/projected/de8c2a71-2563-48f3-a1fc-863201961a2b-kube-api-access-4ntx6\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549672 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-bin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549701 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-conf-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549725 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-ovn\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549748 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzrkz\" (UniqueName: \"kubernetes.io/projected/327e2de0-9591-458f-9728-79adfc2c46eb-kube-api-access-kzrkz\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549790 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/47916393-3c42-4fff-ac7e-3c1a27df4df1-hosts-file\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549813 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-var-lib-kubelet\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549837 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-tuned\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.622399 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549875 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549899 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549921 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25pdt\" (UniqueName: \"kubernetes.io/projected/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kube-api-access-25pdt\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549950 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-cnibin\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549969 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.549986 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysconfig\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550017 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47916393-3c42-4fff-ac7e-3c1a27df4df1-tmp-dir\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550069 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-netns\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550100 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-etc-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550165 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-dbus\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550188 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmtl\" (UniqueName: \"kubernetes.io/projected/5415644a-9ad7-4222-9893-beed7eeb75a7-kube-api-access-4pmtl\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550215 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550236 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-systemd\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550260 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-kubelet\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-systemd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550317 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-kubelet-config\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550351 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-os-release\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550370 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-run\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550393 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-kubelet\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550409 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-multus-certs\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550423 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/327e2de0-9591-458f-9728-79adfc2c46eb-ovn-node-metrics-cert\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-script-lib\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550472 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-serviceca\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.623876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550505 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550550 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-systemd-units\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550584 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-config\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550620 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550650 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-sys-fs\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550679 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv69j\" (UniqueName: \"kubernetes.io/projected/47916393-3c42-4fff-ac7e-3c1a27df4df1-kube-api-access-sv69j\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550704 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-lib-modules\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550730 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g28bz\" (UniqueName: \"kubernetes.io/projected/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-kube-api-access-g28bz\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550774 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-socket-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550828 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-device-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550855 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-os-release\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550895 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-daemon-config\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550915 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4692220-7d1f-4e3d-9dbd-3485cb55e704-konnectivity-ca\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550945 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-k8s-cni-cncf-io\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.624528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.550967 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-sys\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551001 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-hostroot\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551015 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-node-log\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551034 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4692220-7d1f-4e3d-9dbd-3485cb55e704-agent-certs\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551065 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-registration-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-system-cni-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551130 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551178 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-conf\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551221 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cnibin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551238 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7546\" (UniqueName: \"kubernetes.io/projected/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-kube-api-access-k7546\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-slash\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-env-overrides\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17331bcb-03ec-4c32-9fdd-d479965632fb-host-slash\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.625335 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.551359 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-socket-dir-parent\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.626036 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.622436 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12306cecd9bb065dc086ef079ccc1976.slice/crio-35d0a28c2a74bcde572861dd449baa5399e43fc7690a1e946dbacd1e93a0e269 WatchSource:0}: Error finding container 35d0a28c2a74bcde572861dd449baa5399e43fc7690a1e946dbacd1e93a0e269: Status 404 returned error can't find the container with id 35d0a28c2a74bcde572861dd449baa5399e43fc7690a1e946dbacd1e93a0e269 Jul 06 23:48:54.626036 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.624579 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3a72da0cd12d86073bbd691ce262f4.slice/crio-5825e86a530eeaebcec1c1a517809753ce0b3e36163b602a9a3ac9089d8b65af WatchSource:0}: Error finding container 5825e86a530eeaebcec1c1a517809753ce0b3e36163b602a9a3ac9089d8b65af: Status 404 returned error can't find the container with id 5825e86a530eeaebcec1c1a517809753ce0b3e36163b602a9a3ac9089d8b65af Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.627741 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-socket-dir-parent\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651831 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-etc-kubernetes\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651870 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-var-lib-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651885 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-bin\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-host\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651966 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-socket-dir-parent\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651979 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-etc-kubernetes\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.651997 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-var-lib-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652032 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-bin\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652085 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-host\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652075 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652124 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-log-socket\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652148 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.832143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652154 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-log-socket\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652165 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dbj\" (UniqueName: \"kubernetes.io/projected/21436535-c81e-4688-bc63-5c0720cba1cd-kube-api-access-b8dbj\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652234 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cni-binary-copy\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652278 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gpwk\" (UniqueName: \"kubernetes.io/projected/17331bcb-03ec-4c32-9fdd-d479965632fb-kube-api-access-5gpwk\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.652294 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.652367 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.152341979 +0000 UTC m=+2.082399482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-kubernetes\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652406 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-tmp\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-system-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652458 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-netd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652469 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-kubernetes\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652535 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17331bcb-03ec-4c32-9fdd-d479965632fb-iptables-alerter-script\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652556 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-modprobe-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.833270 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652583 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-host\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652605 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652692 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-modprobe-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652669 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-netns\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652694 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-system-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652750 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-multus\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652786 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-host\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ntx6\" (UniqueName: \"kubernetes.io/projected/de8c2a71-2563-48f3-a1fc-863201961a2b-kube-api-access-4ntx6\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652850 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-netns\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652863 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-bin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652864 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652897 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-conf-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652927 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-ovn\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652904 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-multus\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.652962 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzrkz\" (UniqueName: \"kubernetes.io/projected/327e2de0-9591-458f-9728-79adfc2c46eb-kube-api-access-kzrkz\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.834218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653016 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-ovn\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653007 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-conf-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653045 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/47916393-3c42-4fff-ac7e-3c1a27df4df1-hosts-file\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653078 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-var-lib-kubelet\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653077 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-cni-bin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653109 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-tuned\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653036 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653128 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cni-binary-copy\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653131 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/47916393-3c42-4fff-ac7e-3c1a27df4df1-hosts-file\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653138 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-var-lib-kubelet\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653174 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-cni-netd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653168 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653222 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653284 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-cni-dir\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.835188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653318 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25pdt\" (UniqueName: \"kubernetes.io/projected/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kube-api-access-25pdt\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653380 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-cnibin\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653427 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-cnibin\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17331bcb-03ec-4c32-9fdd-d479965632fb-iptables-alerter-script\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653593 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysconfig\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653692 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysconfig\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653725 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653730 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47916393-3c42-4fff-ac7e-3c1a27df4df1-tmp-dir\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653896 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-netns\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653971 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.835977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.653954 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-etc-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654024 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-etc-openvswitch\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654043 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654049 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-netns\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654078 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-dbus\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmtl\" (UniqueName: \"kubernetes.io/projected/5415644a-9ad7-4222-9893-beed7eeb75a7-kube-api-access-4pmtl\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654100 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654129 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47916393-3c42-4fff-ac7e-3c1a27df4df1-tmp-dir\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654161 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654169 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-dbus\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654204 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-systemd\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654237 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-kubelet\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654267 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-systemd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-d\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-kubelet\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.836619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654334 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-systemd\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654350 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-kubelet-config\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654436 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-kubelet-config\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654447 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-run-systemd\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654439 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-os-release\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654492 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-run\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654518 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-kubelet\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654538 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-os-release\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654565 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-var-lib-kubelet\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654558 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-multus-certs\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654596 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5415644a-9ad7-4222-9893-beed7eeb75a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654604 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/327e2de0-9591-458f-9728-79adfc2c46eb-ovn-node-metrics-cert\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654605 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-run\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654640 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-script-lib\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654662 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-serviceca\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.837288 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654665 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-multus-certs\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654679 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654695 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-systemd-units\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654791 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-systemd-units\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654747 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-config\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.654796 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654838 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.654874 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.154851426 +0000 UTC m=+2.084908910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654896 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-sys-fs\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654951 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.654988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv69j\" (UniqueName: \"kubernetes.io/projected/47916393-3c42-4fff-ac7e-3c1a27df4df1-kube-api-access-sv69j\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655021 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-lib-modules\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655025 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-sys-fs\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655186 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g28bz\" (UniqueName: \"kubernetes.io/projected/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-kube-api-access-g28bz\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.837872 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655220 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-socket-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655309 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-device-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655404 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-os-release\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655434 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-daemon-config\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655417 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-serviceca\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4692220-7d1f-4e3d-9dbd-3485cb55e704-konnectivity-ca\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655535 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-device-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655538 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-os-release\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655574 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-k8s-cni-cncf-io\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655576 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-socket-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655625 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-sys\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655647 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-host-run-k8s-cni-cncf-io\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655652 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-hostroot\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655626 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-config\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655680 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-sys\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.838361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655679 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-node-log\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.655722 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-hostroot\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656077 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-lib-modules\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656099 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-node-log\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656375 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4692220-7d1f-4e3d-9dbd-3485cb55e704-agent-certs\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-registration-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656485 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-system-cni-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656592 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-conf\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cnibin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656638 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-multus-daemon-config\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7546\" (UniqueName: \"kubernetes.io/projected/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-kube-api-access-k7546\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-slash\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656758 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-system-cni-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-env-overrides\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.838898 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656824 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4692220-7d1f-4e3d-9dbd-3485cb55e704-konnectivity-ca\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.656871 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17331bcb-03ec-4c32-9fdd-d479965632fb-host-slash\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657050 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17331bcb-03ec-4c32-9fdd-d479965632fb-host-slash\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657086 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-tmp\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657132 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-tuned\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657230 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5415644a-9ad7-4222-9893-beed7eeb75a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657270 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/327e2de0-9591-458f-9728-79adfc2c46eb-host-slash\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657361 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/327e2de0-9591-458f-9728-79adfc2c46eb-ovn-node-metrics-cert\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657412 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-registration-dir\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657550 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-cnibin\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657585 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de8c2a71-2563-48f3-a1fc-863201961a2b-etc-sysctl-conf\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.657788 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-env-overrides\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.658065 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/327e2de0-9591-458f-9728-79adfc2c46eb-ovnkube-script-lib\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.660845 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.660886 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:54.839506 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.660902 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:54.661239 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.161009392 +0000 UTC m=+2.091066898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.661291 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gpwk\" (UniqueName: \"kubernetes.io/projected/17331bcb-03ec-4c32-9fdd-d479965632fb-kube-api-access-5gpwk\") pod \"iptables-alerter-6bw7q\" (UID: \"17331bcb-03ec-4c32-9fdd-d479965632fb\") " pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.662229 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dbj\" (UniqueName: \"kubernetes.io/projected/21436535-c81e-4688-bc63-5c0720cba1cd-kube-api-access-b8dbj\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.662457 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzrkz\" (UniqueName: \"kubernetes.io/projected/327e2de0-9591-458f-9728-79adfc2c46eb-kube-api-access-kzrkz\") pod \"ovnkube-node-4b9rn\" (UID: \"327e2de0-9591-458f-9728-79adfc2c46eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.662545 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4692220-7d1f-4e3d-9dbd-3485cb55e704-agent-certs\") pod \"konnectivity-agent-h9g4p\" (UID: \"c4692220-7d1f-4e3d-9dbd-3485cb55e704\") " pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.662991 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ntx6\" (UniqueName: \"kubernetes.io/projected/de8c2a71-2563-48f3-a1fc-863201961a2b-kube-api-access-4ntx6\") pod \"tuned-2wg8h\" (UID: \"de8c2a71-2563-48f3-a1fc-863201961a2b\") " pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.663698 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25pdt\" (UniqueName: \"kubernetes.io/projected/e8e31f27-3e07-40b6-a523-3613fbb9ad8d-kube-api-access-25pdt\") pod \"aws-ebs-csi-driver-node-xx2xx\" (UID: \"e8e31f27-3e07-40b6-a523-3613fbb9ad8d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.664095 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmtl\" (UniqueName: \"kubernetes.io/projected/5415644a-9ad7-4222-9893-beed7eeb75a7-kube-api-access-4pmtl\") pod \"multus-additional-cni-plugins-cj74h\" (UID: \"5415644a-9ad7-4222-9893-beed7eeb75a7\") " pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.668804 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g28bz\" (UniqueName: \"kubernetes.io/projected/1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38-kube-api-access-g28bz\") pod \"node-ca-h6nbc\" (UID: \"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38\") " pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.675905 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv69j\" (UniqueName: \"kubernetes.io/projected/47916393-3c42-4fff-ac7e-3c1a27df4df1-kube-api-access-sv69j\") pod \"node-resolver-tpbff\" (UID: \"47916393-3c42-4fff-ac7e-3c1a27df4df1\") " pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.675950 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7546\" (UniqueName: \"kubernetes.io/projected/5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24-kube-api-access-k7546\") pod \"multus-vhl7b\" (UID: \"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24\") " pod="openshift-multus/multus-vhl7b" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.789168 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpbff" Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.798573 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47916393_3c42_4fff_ac7e_3c1a27df4df1.slice/crio-1838843b76e6beb46187c658e88e076569d7156697141ee07d0f6c844166e6a8 WatchSource:0}: Error finding container 1838843b76e6beb46187c658e88e076569d7156697141ee07d0f6c844166e6a8: Status 404 returned error can't find the container with id 1838843b76e6beb46187c658e88e076569d7156697141ee07d0f6c844166e6a8 Jul 06 23:48:54.840060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.803900 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6nbc" Jul 06 23:48:54.840533 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.810949 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4e1901_cdb3_4bc4_9c18_4acb64f2aa38.slice/crio-d8043923e4e4366eee2416ee654dc91a3807848e776d2f235d2df05ae609dac9 WatchSource:0}: Error finding container d8043923e4e4366eee2416ee654dc91a3807848e776d2f235d2df05ae609dac9: Status 404 returned error can't find the container with id d8043923e4e4366eee2416ee654dc91a3807848e776d2f235d2df05ae609dac9 Jul 06 23:48:54.840533 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.825863 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cj74h" Jul 06 23:48:54.840533 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.833386 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5415644a_9ad7_4222_9893_beed7eeb75a7.slice/crio-47b151b21868edcf6e5a66c70b501e0ee7a4f0f03e5c4619d99328cd2ba8d0fb WatchSource:0}: Error finding container 47b151b21868edcf6e5a66c70b501e0ee7a4f0f03e5c4619d99328cd2ba8d0fb: Status 404 returned error can't find the container with id 47b151b21868edcf6e5a66c70b501e0ee7a4f0f03e5c4619d99328cd2ba8d0fb Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.847242 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bw7q" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.854872 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17331bcb_03ec_4c32_9fdd_d479965632fb.slice/crio-3b373c8906c6923cc284731610d8e1a601f4c8515ba27aa33fd8189efff5a3b0 WatchSource:0}: Error finding container 3b373c8906c6923cc284731610d8e1a601f4c8515ba27aa33fd8189efff5a3b0: Status 404 returned error can't find the container with id 3b373c8906c6923cc284731610d8e1a601f4c8515ba27aa33fd8189efff5a3b0 Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.867324 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.874441 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e31f27_3e07_40b6_a523_3613fbb9ad8d.slice/crio-0448c9a99795e4254070a6bb69dfc3c2fc2c873cd36df9c2e905503478ebe801 WatchSource:0}: Error finding container 0448c9a99795e4254070a6bb69dfc3c2fc2c873cd36df9c2e905503478ebe801: Status 404 returned error can't find the container with id 0448c9a99795e4254070a6bb69dfc3c2fc2c873cd36df9c2e905503478ebe801 Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.882254 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.889369 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8c2a71_2563_48f3_a1fc_863201961a2b.slice/crio-734eb8e6a8b7ba08880cf9cb204c63ccff6ff0971b1b33e779ed944fafa3e4c9 WatchSource:0}: Error finding container 734eb8e6a8b7ba08880cf9cb204c63ccff6ff0971b1b33e779ed944fafa3e4c9: Status 404 returned error can't find the container with id 734eb8e6a8b7ba08880cf9cb204c63ccff6ff0971b1b33e779ed944fafa3e4c9 Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.905490 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.905524 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vhl7b" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.914691 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.915643 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4692220_7d1f_4e3d_9dbd_3485cb55e704.slice/crio-4839fd31ef0dd3cff81d8e9fb1d4bb8ddb0f2ff3cda58b559466c369bac79d40 WatchSource:0}: Error finding container 4839fd31ef0dd3cff81d8e9fb1d4bb8ddb0f2ff3cda58b559466c369bac79d40: Status 404 returned error can't find the container with id 4839fd31ef0dd3cff81d8e9fb1d4bb8ddb0f2ff3cda58b559466c369bac79d40 Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.916557 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb4b4e8_8c8b_4a4c_8d35_cb4c8fd2ef24.slice/crio-3c1e7f7093eef528313ba5d91de91d66ed16c0de34e960fb4c009ddd582193b7 WatchSource:0}: Error finding container 3c1e7f7093eef528313ba5d91de91d66ed16c0de34e960fb4c009ddd582193b7: Status 404 returned error can't find the container with id 3c1e7f7093eef528313ba5d91de91d66ed16c0de34e960fb4c009ddd582193b7 Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:48:54.924199 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327e2de0_9591_458f_9728_79adfc2c46eb.slice/crio-012eaec4daf6ed71fb70105b581f383f9b8d8b75351fb174bb915decbb1bd9cc WatchSource:0}: Error finding container 012eaec4daf6ed71fb70105b581f383f9b8d8b75351fb174bb915decbb1bd9cc: Status 404 returned error can't find the container with id 012eaec4daf6ed71fb70105b581f383f9b8d8b75351fb174bb915decbb1bd9cc Jul 06 23:48:55.010739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:54.948186 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.161967 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.162007 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.162044 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162139 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162139 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162354 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.162329862 +0000 UTC m=+3.092387358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162154 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162403 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162419 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.162696 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.162461332 +0000 UTC m=+3.092518832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:55.183450 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:55.163162 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:48:56.163151967 +0000 UTC m=+3.093209455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:55.480032 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.460653 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:55.565137 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.523139 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:55 +0000 UTC" deadline="2028-02-25 16:37:15.914037126 +0000 UTC" Jul 06 23:48:55.565137 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.523199 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14368h48m20.390841608s" Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.601056 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.606902 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h9g4p" event={"ID":"c4692220-7d1f-4e3d-9dbd-3485cb55e704","Type":"ContainerStarted","Data":"4839fd31ef0dd3cff81d8e9fb1d4bb8ddb0f2ff3cda58b559466c369bac79d40"} Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.611297 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bw7q" event={"ID":"17331bcb-03ec-4c32-9fdd-d479965632fb","Type":"ContainerStarted","Data":"3b373c8906c6923cc284731610d8e1a601f4c8515ba27aa33fd8189efff5a3b0"} Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.617389 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerStarted","Data":"47b151b21868edcf6e5a66c70b501e0ee7a4f0f03e5c4619d99328cd2ba8d0fb"} Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.630909 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6nbc" event={"ID":"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38","Type":"ContainerStarted","Data":"d8043923e4e4366eee2416ee654dc91a3807848e776d2f235d2df05ae609dac9"} Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.635634 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"1b3a72da0cd12d86073bbd691ce262f4","Type":"ContainerStarted","Data":"5825e86a530eeaebcec1c1a517809753ce0b3e36163b602a9a3ac9089d8b65af"} Jul 06 23:48:55.651589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.641862 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"012eaec4daf6ed71fb70105b581f383f9b8d8b75351fb174bb915decbb1bd9cc"} Jul 06 23:48:55.658628 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.658602 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vhl7b" event={"ID":"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24","Type":"ContainerStarted","Data":"3c1e7f7093eef528313ba5d91de91d66ed16c0de34e960fb4c009ddd582193b7"} Jul 06 23:48:55.673550 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.673517 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" event={"ID":"de8c2a71-2563-48f3-a1fc-863201961a2b","Type":"ContainerStarted","Data":"734eb8e6a8b7ba08880cf9cb204c63ccff6ff0971b1b33e779ed944fafa3e4c9"} Jul 06 23:48:55.683884 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.683528 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" event={"ID":"e8e31f27-3e07-40b6-a523-3613fbb9ad8d","Type":"ContainerStarted","Data":"0448c9a99795e4254070a6bb69dfc3c2fc2c873cd36df9c2e905503478ebe801"} Jul 06 23:48:55.692055 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.691989 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpbff" event={"ID":"47916393-3c42-4fff-ac7e-3c1a27df4df1","Type":"ContainerStarted","Data":"1838843b76e6beb46187c658e88e076569d7156697141ee07d0f6c844166e6a8"} Jul 06 23:48:55.700176 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:55.700147 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" event={"ID":"12306cecd9bb065dc086ef079ccc1976","Type":"ContainerStarted","Data":"35d0a28c2a74bcde572861dd449baa5399e43fc7690a1e946dbacd1e93a0e269"} Jul 06 23:48:56.173482 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.173445 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:56.173667 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.173513 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:56.173667 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.173556 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:56.173739 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.173705 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:56.173780 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.173771 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:58.173752826 +0000 UTC m=+5.103810313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:56.174269 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174245 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.174389 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174313 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:48:58.174294826 +0000 UTC m=+5.104352311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.174459 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174407 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:56.174459 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174422 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:56.174459 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174436 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:56.174614 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.174483 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:48:58.174459765 +0000 UTC m=+5.104517266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:56.524296 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.524201 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:55 +0000 UTC" deadline="2028-03-27 10:02:40.573218662 +0000 UTC" Jul 06 23:48:56.524296 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.524244 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15106h13m44.048982205s" Jul 06 23:48:56.584044 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.584009 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:56.584212 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.584161 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:48:56.584284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.584168 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:56.584369 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.584348 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:48:56.584429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.584223 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:56.584482 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:56.584466 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:48:56.855324 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:56.855223 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:58.191642 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.191550 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.191641 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.191684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.191855 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.191926 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:02.191906734 +0000 UTC m=+9.121964235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192377 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:58.192484 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192435 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:02.192418124 +0000 UTC m=+9.122475623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:58.192997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192519 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:58.192997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192534 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:58.192997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192547 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:58.192997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.192587 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:49:02.192573992 +0000 UTC m=+9.122631491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.583435 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.583585 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.584041 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.584145 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:48:58.584210 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:48:58.584402 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:48:58.584299 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:00.584084 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:00.584029 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:00.584589 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:00.584165 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:00.584589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:00.584239 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:00.584589 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:00.584299 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:00.584589 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:00.584338 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:00.584589 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:00.584426 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:02.228743 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.228703 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.228790 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.228844 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.228989 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229052 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:10.229033169 +0000 UTC m=+17.159090651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229140 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229154 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229166 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:02.229228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229207 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:49:10.229188643 +0000 UTC m=+17.159246131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:02.229627 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229262 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:02.229627 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.229293 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:10.229282875 +0000 UTC m=+17.159340361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:02.583601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.583526 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:02.583601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.583551 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:02.583826 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.583671 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:02.583826 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:02.583803 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:02.583936 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.583891 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:02.583936 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:02.583732 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:04.583670 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:04.583690 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:04.583815 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:04.583888 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:04.584004 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:04.584194 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:04.584084 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:06.583516 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:06.583485 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:06.584011 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:06.583527 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:06.584011 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:06.583526 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:06.584011 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:06.583604 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:06.584011 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:06.583695 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:06.584011 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:06.583808 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:08.584007 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:08.583973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:08.584457 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:08.584021 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:08.584457 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:08.584089 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:08.584457 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:08.584100 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:08.584457 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:08.584211 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:08.584457 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:08.584301 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:10.291888 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.291845 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.291993 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292064 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:26.29204554 +0000 UTC m=+33.222103039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.291999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292101 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.292149 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292158 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:26.292139648 +0000 UTC m=+33.222197132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292243 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292258 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292270 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:10.292356 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.292318 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:49:26.292306626 +0000 UTC m=+33.222364145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:10.584048 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.583973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:10.584228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.583976 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:10.584228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.584098 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:10.584228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:10.583977 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:10.584228 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.584151 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:10.584392 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:10.584267 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:12.583204 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:12.583164 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:12.583204 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:12.583199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:12.583664 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:12.583199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:12.583664 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:12.583275 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:12.583664 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:12.583354 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:12.583664 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:12.583459 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:14.583762 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:14.583721 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:14.583762 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:14.583759 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:14.584254 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:14.583852 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:14.584254 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:14.583868 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:14.584254 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:14.584026 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:14.584254 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:14.584116 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:15.764781 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.764585 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"6d4982823687a86fd90e9ac407f22a1bbf8700642a46ee368c0c325d1621b952"} Jul 06 23:49:15.766276 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.766250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vhl7b" event={"ID":"5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24","Type":"ContainerStarted","Data":"3203c11d7c999095b6cea7bad8b4d9394591b239a0dc1b4d555b67ccdf4724c2"} Jul 06 23:49:15.767772 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.767735 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" event={"ID":"de8c2a71-2563-48f3-a1fc-863201961a2b","Type":"ContainerStarted","Data":"64b74b70bc3c9a7e8d154d4aa967fccd2bef46b63d4f8ffa5057f8691ad26c73"} Jul 06 23:49:15.769202 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.769175 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" event={"ID":"12306cecd9bb065dc086ef079ccc1976","Type":"ContainerStarted","Data":"630291955dbfc91f48a554aa29c87bae8e931e7e7a06ba721704c0e86a48e59c"} Jul 06 23:49:15.784688 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.784647 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vhl7b" podStartSLOduration=1.155342255 podStartE2EDuration="21.784635757s" totalImagesPullingTime="20.629293502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:15.78420622 +0000 UTC m=+22.714263716" watchObservedRunningTime="2026-07-06 23:49:15.784635757 +0000 UTC m=+22.714693261" Jul 06 23:49:15.799263 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.799223 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-169.ec2.internal" podStartSLOduration=21.799210001 podStartE2EDuration="21.799210001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:15.799141108 +0000 UTC m=+22.729198664" watchObservedRunningTime="2026-07-06 23:49:15.799210001 +0000 UTC m=+22.729267543" Jul 06 23:49:15.818128 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:15.818079 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2wg8h" podStartSLOduration=1.197568773 podStartE2EDuration="21.818062385s" totalImagesPullingTime="20.620493612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:15.817333861 +0000 UTC m=+22.747391365" watchObservedRunningTime="2026-07-06 23:49:15.818062385 +0000 UTC m=+22.748119890" Jul 06 23:49:16.583665 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.583506 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:16.583793 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.583545 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:16.583793 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:16.583722 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:16.583793 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.583551 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:16.583997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:16.583904 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:16.584058 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:16.584021 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:16.772683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.772596 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h9g4p" event={"ID":"c4692220-7d1f-4e3d-9dbd-3485cb55e704","Type":"ContainerStarted","Data":"fb86b9721e327a1975faf6593f6cf61234a07f7cee8af0bc9c9e7b332f0a31f0"} Jul 06 23:49:16.774159 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.774137 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bw7q" event={"ID":"17331bcb-03ec-4c32-9fdd-d479965632fb","Type":"ContainerStarted","Data":"d402ed375280d9a4aea87d51e5f609392f6f602a0ecf5beda3bd923fdebccba5"} Jul 06 23:49:16.775597 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.775574 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="fe45653f2c620caee8036bc9af270adaf817a02d52d6bab5dc6e26e8ac09d254" exitCode=0 Jul 06 23:49:16.775678 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.775645 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"fe45653f2c620caee8036bc9af270adaf817a02d52d6bab5dc6e26e8ac09d254"} Jul 06 23:49:16.777109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.777079 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6nbc" event={"ID":"1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38","Type":"ContainerStarted","Data":"52387efa91f45c7d3625f387764aa1028d37760f5870378159e5bee0414791c0"} Jul 06 23:49:16.778557 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.778539 2556 generic.go:356] "Generic (PLEG): container finished" podID="1b3a72da0cd12d86073bbd691ce262f4" containerID="e834c49a062540588a4dc239f8ff4c5d37e93e40f3d8ec3f06c17dbe2a79e0ed" exitCode=0 Jul 06 23:49:16.778646 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.778630 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"1b3a72da0cd12d86073bbd691ce262f4","Type":"ContainerDied","Data":"e834c49a062540588a4dc239f8ff4c5d37e93e40f3d8ec3f06c17dbe2a79e0ed"} Jul 06 23:49:16.781352 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781337 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:49:16.781677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781658 2556 generic.go:356] "Generic (PLEG): container finished" podID="327e2de0-9591-458f-9728-79adfc2c46eb" containerID="51efb19683d72205b2b8cd076815e9fbacf0ed10ccfe7fb6a16b63f9b4002099" exitCode=1 Jul 06 23:49:16.781740 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781724 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"4b1b78056179852d88eb0317e7336ec7df99fa10ce66ab1cd2b763b6f25f493d"} Jul 06 23:49:16.781818 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781794 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"76bf98b25abf91edca78be2771f93d7fc6d83add85df559b69ca6d725dfb1be9"} Jul 06 23:49:16.781875 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781826 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"a88b0cbe2a1c230619a0d93859d3d045a76d26442431450ef8bf01a6fe9b3998"} Jul 06 23:49:16.781875 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781866 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"5e4889acd1165619246cf2e63ba0f1a6f01b505c75f9f2536ea09a402c4d9046"} Jul 06 23:49:16.781965 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.781900 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerDied","Data":"51efb19683d72205b2b8cd076815e9fbacf0ed10ccfe7fb6a16b63f9b4002099"} Jul 06 23:49:16.783113 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.783094 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" event={"ID":"e8e31f27-3e07-40b6-a523-3613fbb9ad8d","Type":"ContainerStarted","Data":"cf61e5dcb91452472c7e4c2a99c243ee2361f09d162ec219f79bf6f193d22a2b"} Jul 06 23:49:16.784354 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.784337 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpbff" event={"ID":"47916393-3c42-4fff-ac7e-3c1a27df4df1","Type":"ContainerStarted","Data":"3ce3415617f5788ae7a153650d97ce14e1f05d3fd673a94bcef4c1cc804bf652"} Jul 06 23:49:16.788313 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.788276 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h9g4p" podStartSLOduration=2.218159411 podStartE2EDuration="22.788264319s" totalImagesPullingTime="20.570104908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:16.78807607 +0000 UTC m=+23.718133565" watchObservedRunningTime="2026-07-06 23:49:16.788264319 +0000 UTC m=+23.718321823" Jul 06 23:49:16.803130 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.803096 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tpbff" podStartSLOduration=2.093067738 podStartE2EDuration="22.803083985s" totalImagesPullingTime="20.710016247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:16.803015092 +0000 UTC m=+23.733072610" watchObservedRunningTime="2026-07-06 23:49:16.803083985 +0000 UTC m=+23.733141489" Jul 06 23:49:16.819170 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.818289 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h6nbc" podStartSLOduration=6.392305053 podStartE2EDuration="22.818274079s" totalImagesPullingTime="16.425969026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:16.817681612 +0000 UTC m=+23.747739120" watchObservedRunningTime="2026-07-06 23:49:16.818274079 +0000 UTC m=+23.748331586" Jul 06 23:49:16.858402 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:16.858358 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6bw7q" podStartSLOduration=2.241465342 podStartE2EDuration="22.858344515s" totalImagesPullingTime="20.616879173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:16.858231395 +0000 UTC m=+23.788288899" watchObservedRunningTime="2026-07-06 23:49:16.858344515 +0000 UTC m=+23.788402019" Jul 06 23:49:17.788312 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:17.788269 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" event={"ID":"1b3a72da0cd12d86073bbd691ce262f4","Type":"ContainerStarted","Data":"4cba8070a5e8d3e1a8c00f3e3333cad2bbfd8bb0c4c820d5784260684969ca94"} Jul 06 23:49:17.806155 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:17.806078 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-169.ec2.internal" podStartSLOduration=23.806062685 podStartE2EDuration="23.806062685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.805800682 +0000 UTC m=+24.735858187" watchObservedRunningTime="2026-07-06 23:49:17.806062685 +0000 UTC m=+24.736120188" Jul 06 23:49:18.144550 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.144503 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:49:18.544731 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.544565 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:49:18.144547726Z","UUID":"b6a3b4d6-b6ff-45c8-9355-e38811c736ce","Handler":null,"Name":"","Endpoint":""} Jul 06 23:49:18.546307 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.546282 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:49:18.546442 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.546328 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:49:18.584016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.583956 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:18.584016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.583987 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:18.584224 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.583987 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:18.584224 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:18.584076 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:18.584224 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:18.584161 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:18.584337 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:18.584246 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:18.794548 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.794514 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:49:18.795149 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.795018 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"625961a5f3f8ced84e68d4905b38473a9b3eac47c433c0d69ad2ecd4fa09344d"} Jul 06 23:49:18.797614 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:18.797588 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" event={"ID":"e8e31f27-3e07-40b6-a523-3613fbb9ad8d","Type":"ContainerStarted","Data":"a1a5bf39a79af2d50a6743f176e279df27d182320ab492546acbf04eda3114cd"} Jul 06 23:49:19.802273 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:19.802233 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" event={"ID":"e8e31f27-3e07-40b6-a523-3613fbb9ad8d","Type":"ContainerStarted","Data":"ebc723b754f8729349065ddb5034f1030ff0b7c1b6be820b7aff35fb37759163"} Jul 06 23:49:19.823968 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:19.823922 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xx2xx" podStartSLOduration=2.743690553 podStartE2EDuration="25.823908108s" totalImagesPullingTime="23.080217555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:19.823569457 +0000 UTC m=+26.753626973" watchObservedRunningTime="2026-07-06 23:49:19.823908108 +0000 UTC m=+26.753965611" Jul 06 23:49:20.583971 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:20.583936 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:20.584158 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:20.583936 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:20.584158 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:20.584067 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:20.584158 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:20.584104 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:20.584317 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:20.584205 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:20.584317 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:20.584267 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:20.938280 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:20.938235 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:49:20.938939 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:20.938912 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:49:21.808372 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.808192 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="4246933434ed4a97bb4dbbb05355fcadcb68f7ec2d20d1c5b6d92d1d58eb792d" exitCode=0 Jul 06 23:49:21.808534 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.808270 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"4246933434ed4a97bb4dbbb05355fcadcb68f7ec2d20d1c5b6d92d1d58eb792d"} Jul 06 23:49:21.812086 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812070 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:49:21.812419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812395 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"c20d13990d356e63134cd41a90ee90461723255c9619c06295c1c562cdfa92a4"} Jul 06 23:49:21.812678 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812650 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:49:21.812802 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812688 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:21.812802 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812739 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:21.812948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812810 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:21.812948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.812854 2556 scope.go:117] "RemoveContainer" containerID="51efb19683d72205b2b8cd076815e9fbacf0ed10ccfe7fb6a16b63f9b4002099" Jul 06 23:49:21.813593 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.813549 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h9g4p" Jul 06 23:49:21.828634 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.828561 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:21.830444 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:21.830426 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:22.584212 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.584176 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:22.585040 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.584176 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:22.585040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:22.584328 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:22.585040 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.584176 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:22.585040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:22.584393 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:22.585040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:22.584494 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:22.818910 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.818882 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:49:22.819361 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.819332 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" event={"ID":"327e2de0-9591-458f-9728-79adfc2c46eb","Type":"ContainerStarted","Data":"dc5f223551cb427d985c914b27e34c8a6e7e6ea00fb5f426566d6ab1f272f64c"} Jul 06 23:49:22.848891 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:22.848805 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" podStartSLOduration=8.174411512 podStartE2EDuration="28.848782703s" totalImagesPullingTime="20.674371191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:22.847303717 +0000 UTC m=+29.777361220" watchObservedRunningTime="2026-07-06 23:49:22.848782703 +0000 UTC m=+29.778840214" Jul 06 23:49:23.301184 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.300970 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xjzp"] Jul 06 23:49:23.301416 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.301288 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:23.301483 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:23.301415 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:23.304408 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.304384 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hhnlw"] Jul 06 23:49:23.304517 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.304464 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:23.304563 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:23.304544 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:23.305195 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.305174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckczt"] Jul 06 23:49:23.305304 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.305253 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:23.305366 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:23.305338 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:23.824237 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.824200 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="2d45962b69f9b3c3d87f53849981caf1ae8d9ae07be34a9ec8367f680c68e54c" exitCode=0 Jul 06 23:49:23.824686 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:23.824287 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"2d45962b69f9b3c3d87f53849981caf1ae8d9ae07be34a9ec8367f680c68e54c"} Jul 06 23:49:24.583830 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:24.583793 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:24.583997 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:24.583946 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:25.583696 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:25.583621 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:25.583696 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:25.583654 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:25.584190 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:25.583733 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:25.584190 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:25.583851 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:25.833831 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:25.833733 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="957042e108e23394073e63bc9d3880971dbe1a1d7434ade74946299876ab4a1d" exitCode=0 Jul 06 23:49:25.833831 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:25.833790 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"957042e108e23394073e63bc9d3880971dbe1a1d7434ade74946299876ab4a1d"} Jul 06 23:49:26.317322 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:26.317293 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:26.317334 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:26.317360 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317430 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317447 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317447 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317462 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:49:26.317467 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317470 2556 projected.go:196] Error preparing data for projected volume kube-api-access-p6hx6 for pod openshift-network-diagnostics/network-check-target-hhnlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:26.317863 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317489 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret podName:283f81b9-52b3-482e-9f4b-6bf4cf1c09f2 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:58.317475357 +0000 UTC m=+65.247532839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret") pod "global-pull-secret-syncer-ckczt" (UID: "283f81b9-52b3-482e-9f4b-6bf4cf1c09f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:26.317863 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317504 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs podName:21436535-c81e-4688-bc63-5c0720cba1cd nodeName:}" failed. No retries permitted until 2026-07-06 23:49:58.317497573 +0000 UTC m=+65.247555054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs") pod "network-metrics-daemon-7xjzp" (UID: "21436535-c81e-4688-bc63-5c0720cba1cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:26.317863 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.317513 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6 podName:4312a9f3-c7a4-4382-bcae-ff87dd8379ef nodeName:}" failed. No retries permitted until 2026-07-06 23:49:58.317508482 +0000 UTC m=+65.247565965 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p6hx6" (UniqueName: "kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6") pod "network-check-target-hhnlw" (UID: "4312a9f3-c7a4-4382-bcae-ff87dd8379ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:26.583686 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:26.583616 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:26.583841 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:26.583758 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckczt" podUID="283f81b9-52b3-482e-9f4b-6bf4cf1c09f2" Jul 06 23:49:27.583816 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:27.583635 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:27.583816 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:27.583774 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xjzp" podUID="21436535-c81e-4688-bc63-5c0720cba1cd" Jul 06 23:49:27.583816 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:27.583818 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:27.584367 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:27.583909 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hhnlw" podUID="4312a9f3-c7a4-4382-bcae-ff87dd8379ef" Jul 06 23:49:28.322956 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.322713 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-169.ec2.internal" event="NodeReady" Jul 06 23:49:28.323116 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.323033 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:49:28.373453 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.373420 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mn5vl"] Jul 06 23:49:28.375834 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.375498 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.375971 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.375934 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wmgj8"] Jul 06 23:49:28.377647 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.377628 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.378142 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.378124 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:49:28.378477 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.378464 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k5jpj\"" Jul 06 23:49:28.378522 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.378506 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:49:28.382489 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.382468 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:49:28.382606 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.382542 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w55hv\"" Jul 06 23:49:28.382606 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.382576 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:49:28.382773 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.382740 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:49:28.388151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.388129 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mn5vl"] Jul 06 23:49:28.389607 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.389424 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wmgj8"] Jul 06 23:49:28.534804 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392b3d1-2fbc-4704-9f40-60f4d730adf7-cert\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.534974 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73de84aa-74d2-4217-9e31-d069995b0a3a-config-volume\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.534974 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534871 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpnh\" (UniqueName: \"kubernetes.io/projected/2392b3d1-2fbc-4704-9f40-60f4d730adf7-kube-api-access-8kpnh\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.534974 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534896 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73de84aa-74d2-4217-9e31-d069995b0a3a-tmp-dir\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.534974 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534919 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfl4r\" (UniqueName: \"kubernetes.io/projected/73de84aa-74d2-4217-9e31-d069995b0a3a-kube-api-access-gfl4r\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.534974 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.534952 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.583879 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.583807 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:28.586546 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.586519 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:28.635635 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635604 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392b3d1-2fbc-4704-9f40-60f4d730adf7-cert\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.635817 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73de84aa-74d2-4217-9e31-d069995b0a3a-config-volume\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.635944 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635801 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpnh\" (UniqueName: \"kubernetes.io/projected/2392b3d1-2fbc-4704-9f40-60f4d730adf7-kube-api-access-8kpnh\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.635944 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635852 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73de84aa-74d2-4217-9e31-d069995b0a3a-tmp-dir\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.635944 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635883 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfl4r\" (UniqueName: \"kubernetes.io/projected/73de84aa-74d2-4217-9e31-d069995b0a3a-kube-api-access-gfl4r\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.636148 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.635946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.636148 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:28.636060 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:28.636206 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:28.636147 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:29.136113597 +0000 UTC m=+36.066171097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:28.636331 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.636281 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73de84aa-74d2-4217-9e31-d069995b0a3a-config-volume\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.636505 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.636482 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73de84aa-74d2-4217-9e31-d069995b0a3a-tmp-dir\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.640695 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.640676 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392b3d1-2fbc-4704-9f40-60f4d730adf7-cert\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.643967 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.643949 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfl4r\" (UniqueName: \"kubernetes.io/projected/73de84aa-74d2-4217-9e31-d069995b0a3a-kube-api-access-gfl4r\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:28.644283 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.644259 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpnh\" (UniqueName: \"kubernetes.io/projected/2392b3d1-2fbc-4704-9f40-60f4d730adf7-kube-api-access-8kpnh\") pod \"ingress-canary-wmgj8\" (UID: \"2392b3d1-2fbc-4704-9f40-60f4d730adf7\") " pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.696219 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.696189 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wmgj8" Jul 06 23:49:28.828209 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.828174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wmgj8"] Jul 06 23:49:28.832495 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:28.832469 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2392b3d1_2fbc_4704_9f40_60f4d730adf7.slice/crio-32a172aa7a4a2b9c1fd8b08f36972b2626799608e8ae03463ee0a5f71ef51220 WatchSource:0}: Error finding container 32a172aa7a4a2b9c1fd8b08f36972b2626799608e8ae03463ee0a5f71ef51220: Status 404 returned error can't find the container with id 32a172aa7a4a2b9c1fd8b08f36972b2626799608e8ae03463ee0a5f71ef51220 Jul 06 23:49:28.842300 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:28.842274 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wmgj8" event={"ID":"2392b3d1-2fbc-4704-9f40-60f4d730adf7","Type":"ContainerStarted","Data":"32a172aa7a4a2b9c1fd8b08f36972b2626799608e8ae03463ee0a5f71ef51220"} Jul 06 23:49:29.141010 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.140923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:29.141159 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:29.141079 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:29.141159 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:29.141152 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:30.141135253 +0000 UTC m=+37.071192755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:29.583692 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.583661 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:29.583886 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.583661 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:29.587016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.586834 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:29.587016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.586847 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dwstx\"" Jul 06 23:49:29.587016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.586870 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:29.587016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.586925 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:29.587994 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:29.587978 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzg9j\"" Jul 06 23:49:30.150088 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.150048 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:30.150284 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:30.150197 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:30.150284 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:30.150275 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:32.150253469 +0000 UTC m=+39.080310975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:30.857002 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.856973 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kjm87"] Jul 06 23:49:30.859504 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.859478 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4"] Jul 06 23:49:30.859622 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.859605 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.862387 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz"] Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.862607 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.862893 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.862904 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.863346 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.863726 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cldcg\"" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.864024 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.864260 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j68w9"] Jul 06 23:49:30.864658 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.864604 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:30.866958 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.866939 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qdx26\"" Jul 06 23:49:30.867234 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.867191 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.867357 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.867283 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.869507 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.869487 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.871681 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.871656 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 23:49:30.872345 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.872326 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 23:49:30.872650 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.872620 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.873073 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.873055 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.873204 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.873185 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4"] Jul 06 23:49:30.873264 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.873242 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 23:49:30.873313 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.873297 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hr8jp\"" Jul 06 23:49:30.874257 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.874227 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 23:49:30.875089 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.875071 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 23:49:30.876100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.876078 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.876214 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.876193 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.876322 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.876087 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ftqhh\"" Jul 06 23:49:30.886144 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.886123 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kjm87"] Jul 06 23:49:30.887642 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.887592 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j68w9"] Jul 06 23:49:30.888490 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.888447 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz"] Jul 06 23:49:30.890304 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.890282 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 23:49:30.955017 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.954978 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9b6988657-sb8wz"] Jul 06 23:49:30.956356 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956320 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb76l\" (UniqueName: \"kubernetes.io/projected/cd94558a-fa48-47f1-b582-d14568f0d0a4-kube-api-access-xb76l\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:30.956495 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956428 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-config\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:30.956558 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956495 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-tmp\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.956558 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b60ae206-9e76-4030-8791-5e6975a48b7d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:30.956661 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956554 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwh28\" (UniqueName: \"kubernetes.io/projected/b60ae206-9e76-4030-8791-5e6975a48b7d-kube-api-access-nwh28\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:30.956661 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956606 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-snapshots\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.956661 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60ae206-9e76-4030-8791-5e6975a48b7d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:30.956843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956671 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd94558a-fa48-47f1-b582-d14568f0d0a4-serving-cert\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:30.956843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956723 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-trusted-ca\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:30.956843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956776 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.956843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956808 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9j2\" (UniqueName: \"kubernetes.io/projected/983d417b-c423-4a1f-b287-bfc925dcf1c0-kube-api-access-5j9j2\") pod \"volume-data-source-validator-5d456fdbcd-77dm4\" (UID: \"983d417b-c423-4a1f-b287-bfc925dcf1c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" Jul 06 23:49:30.956843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dca4762-89a1-4978-aef9-09093526d95e-serving-cert\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.957068 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqwpt\" (UniqueName: \"kubernetes.io/projected/0dca4762-89a1-4978-aef9-09093526d95e-kube-api-access-pqwpt\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.957068 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956948 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:30.957068 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.956981 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:30.960853 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.960831 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 23:49:30.961090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.961070 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.961459 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.961438 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 23:49:30.962006 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.961674 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-crbjf\"" Jul 06 23:49:30.962006 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.961717 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 23:49:30.962006 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.961883 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.965696 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.965657 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 23:49:30.989178 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:30.989145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9b6988657-sb8wz"] Jul 06 23:49:31.057612 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057569 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-snapshots\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.057612 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057613 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60ae206-9e76-4030-8791-5e6975a48b7d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd94558a-fa48-47f1-b582-d14568f0d0a4-serving-cert\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057678 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-trusted-ca\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057724 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9j2\" (UniqueName: \"kubernetes.io/projected/983d417b-c423-4a1f-b287-bfc925dcf1c0-kube-api-access-5j9j2\") pod \"volume-data-source-validator-5d456fdbcd-77dm4\" (UID: \"983d417b-c423-4a1f-b287-bfc925dcf1c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057774 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-stats-auth\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dca4762-89a1-4978-aef9-09093526d95e-serving-cert\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057843 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqwpt\" (UniqueName: \"kubernetes.io/projected/0dca4762-89a1-4978-aef9-09093526d95e-kube-api-access-pqwpt\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.057874 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057868 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057897 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g824z\" (UniqueName: \"kubernetes.io/projected/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-kube-api-access-g824z\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057934 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-default-certificate\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.057957 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058009 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058042 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb76l\" (UniqueName: \"kubernetes.io/projected/cd94558a-fa48-47f1-b582-d14568f0d0a4-kube-api-access-xb76l\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-config\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058132 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-tmp\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058158 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b60ae206-9e76-4030-8791-5e6975a48b7d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.058297 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058187 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwh28\" (UniqueName: \"kubernetes.io/projected/b60ae206-9e76-4030-8791-5e6975a48b7d-kube-api-access-nwh28\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.058679 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.058297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-snapshots\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.060442 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.060981 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b60ae206-9e76-4030-8791-5e6975a48b7d-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061003 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dca4762-89a1-4978-aef9-09093526d95e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061009 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dca4762-89a1-4978-aef9-09093526d95e-tmp\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061065 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd94558a-fa48-47f1-b582-d14568f0d0a4-serving-cert\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061276 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-trusted-ca\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061541 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b60ae206-9e76-4030-8791-5e6975a48b7d-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.064151 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.061688 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd94558a-fa48-47f1-b582-d14568f0d0a4-config\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.065489 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.065446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dca4762-89a1-4978-aef9-09093526d95e-serving-cert\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.066882 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.066860 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwh28\" (UniqueName: \"kubernetes.io/projected/b60ae206-9e76-4030-8791-5e6975a48b7d-kube-api-access-nwh28\") pod \"kube-storage-version-migrator-operator-846bf668f7-w5vzz\" (UID: \"b60ae206-9e76-4030-8791-5e6975a48b7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.067643 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.067620 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqwpt\" (UniqueName: \"kubernetes.io/projected/0dca4762-89a1-4978-aef9-09093526d95e-kube-api-access-pqwpt\") pod \"insights-operator-5c847dcfc4-j68w9\" (UID: \"0dca4762-89a1-4978-aef9-09093526d95e\") " pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.069319 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.069297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9j2\" (UniqueName: \"kubernetes.io/projected/983d417b-c423-4a1f-b287-bfc925dcf1c0-kube-api-access-5j9j2\") pod \"volume-data-source-validator-5d456fdbcd-77dm4\" (UID: \"983d417b-c423-4a1f-b287-bfc925dcf1c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" Jul 06 23:49:31.069860 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.069843 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb76l\" (UniqueName: \"kubernetes.io/projected/cd94558a-fa48-47f1-b582-d14568f0d0a4-kube-api-access-xb76l\") pod \"console-operator-68b757fb67-kjm87\" (UID: \"cd94558a-fa48-47f1-b582-d14568f0d0a4\") " pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.158657 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.158563 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-stats-auth\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.158657 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.158607 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.158657 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.158634 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g824z\" (UniqueName: \"kubernetes.io/projected/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-kube-api-access-g824z\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.158957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.158663 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-default-certificate\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.158957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.158687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.158957 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.158835 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:31.658814587 +0000 UTC m=+38.588872089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:31.158957 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.158906 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 23:49:31.158957 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.158954 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:31.658940644 +0000 UTC m=+38.588998143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : secret "router-metrics-certs-default" not found Jul 06 23:49:31.161523 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.161493 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-stats-auth\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.161523 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.161509 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-default-certificate\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.168409 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.168382 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g824z\" (UniqueName: \"kubernetes.io/projected/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-kube-api-access-g824z\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.172520 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.172498 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:31.181825 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.181801 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" Jul 06 23:49:31.193582 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.193564 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" Jul 06 23:49:31.195110 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.195096 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" Jul 06 23:49:31.664109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.663905 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.664109 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.664107 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:32.664084361 +0000 UTC m=+39.594141847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:31.664342 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:31.664157 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:31.664342 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.664293 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 23:49:31.664342 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:31.664334 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:32.664323944 +0000 UTC m=+39.594381444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : secret "router-metrics-certs-default" not found Jul 06 23:49:32.168736 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.167926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:32.168736 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:32.168110 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:32.168736 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:32.168182 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:36.168164377 +0000 UTC m=+43.098221881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:32.282740 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.282711 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-j68w9"] Jul 06 23:49:32.288101 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:32.288070 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dca4762_89a1_4978_aef9_09093526d95e.slice/crio-9a664c11ebe7d84bc93da616900fe8ff143333783ca155de8967ce36a01f5951 WatchSource:0}: Error finding container 9a664c11ebe7d84bc93da616900fe8ff143333783ca155de8967ce36a01f5951: Status 404 returned error can't find the container with id 9a664c11ebe7d84bc93da616900fe8ff143333783ca155de8967ce36a01f5951 Jul 06 23:49:32.290506 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.290256 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4"] Jul 06 23:49:32.293256 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.293194 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz"] Jul 06 23:49:32.294649 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:32.294554 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983d417b_c423_4a1f_b287_bfc925dcf1c0.slice/crio-cc4dde6d90ab083a4b6f9c780f80ae2453c8be305223b559996b7690c23aa70b WatchSource:0}: Error finding container cc4dde6d90ab083a4b6f9c780f80ae2453c8be305223b559996b7690c23aa70b: Status 404 returned error can't find the container with id cc4dde6d90ab083a4b6f9c780f80ae2453c8be305223b559996b7690c23aa70b Jul 06 23:49:32.296934 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.296872 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kjm87"] Jul 06 23:49:32.302183 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:32.301264 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd94558a_fa48_47f1_b582_d14568f0d0a4.slice/crio-370c84030e0b3480aa4e915aa0beb7c99a74ff8f49f5641db8ab8dd5ad7de1b9 WatchSource:0}: Error finding container 370c84030e0b3480aa4e915aa0beb7c99a74ff8f49f5641db8ab8dd5ad7de1b9: Status 404 returned error can't find the container with id 370c84030e0b3480aa4e915aa0beb7c99a74ff8f49f5641db8ab8dd5ad7de1b9 Jul 06 23:49:32.672226 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.671953 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:32.672226 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:32.672127 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:34.672108534 +0000 UTC m=+41.602166030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:32.672226 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.672217 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:32.672494 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:32.672385 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 23:49:32.672494 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:32.672474 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:34.672451307 +0000 UTC m=+41.602508807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : secret "router-metrics-certs-default" not found Jul 06 23:49:32.854377 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.854344 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" event={"ID":"b60ae206-9e76-4030-8791-5e6975a48b7d","Type":"ContainerStarted","Data":"03731c80d80b8ee562e68f076f037a6e799a690aeddc897a20040932b9ff9a81"} Jul 06 23:49:32.855417 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.855385 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" event={"ID":"983d417b-c423-4a1f-b287-bfc925dcf1c0","Type":"ContainerStarted","Data":"cc4dde6d90ab083a4b6f9c780f80ae2453c8be305223b559996b7690c23aa70b"} Jul 06 23:49:32.856410 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.856386 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" event={"ID":"cd94558a-fa48-47f1-b582-d14568f0d0a4","Type":"ContainerStarted","Data":"370c84030e0b3480aa4e915aa0beb7c99a74ff8f49f5641db8ab8dd5ad7de1b9"} Jul 06 23:49:32.857434 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.857411 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" event={"ID":"0dca4762-89a1-4978-aef9-09093526d95e","Type":"ContainerStarted","Data":"9a664c11ebe7d84bc93da616900fe8ff143333783ca155de8967ce36a01f5951"} Jul 06 23:49:32.860124 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.860097 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="0873d0576dedf6ed864d77bc07a70277f2954b74283adb7d40e9586c1ac9318e" exitCode=0 Jul 06 23:49:32.860224 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.860173 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"0873d0576dedf6ed864d77bc07a70277f2954b74283adb7d40e9586c1ac9318e"} Jul 06 23:49:32.861690 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.861644 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wmgj8" event={"ID":"2392b3d1-2fbc-4704-9f40-60f4d730adf7","Type":"ContainerStarted","Data":"d6af72a82efcf0650367b0037aa38e3d60ebe6e569130488eebd3ba48e6edd8d"} Jul 06 23:49:32.902613 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:32.902391 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wmgj8" podStartSLOduration=1.629894525 podStartE2EDuration="4.902373127s" totalImagesPullingTime="3.272478602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:32.901732973 +0000 UTC m=+39.831790478" watchObservedRunningTime="2026-07-06 23:49:32.902373127 +0000 UTC m=+39.832430633" Jul 06 23:49:33.870869 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:33.870829 2556 generic.go:356] "Generic (PLEG): container finished" podID="5415644a-9ad7-4222-9893-beed7eeb75a7" containerID="972e74a9037485bc428f1f12962d054e329c941224c020f644c2a7bd419fadae" exitCode=0 Jul 06 23:49:33.871435 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:33.870952 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerDied","Data":"972e74a9037485bc428f1f12962d054e329c941224c020f644c2a7bd419fadae"} Jul 06 23:49:34.690579 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:34.690545 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:34.690802 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:34.690700 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 23:49:34.690802 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:34.690775 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:34.690802 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:34.690788 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.690771681 +0000 UTC m=+45.620829167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : secret "router-metrics-certs-default" not found Jul 06 23:49:34.690984 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:34.690934 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.690917275 +0000 UTC m=+45.620974775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:34.877228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:34.877196 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cj74h" event={"ID":"5415644a-9ad7-4222-9893-beed7eeb75a7","Type":"ContainerStarted","Data":"903392a2d6f967a7fc66ee3685e3ecdab607f416051ce990d653b4979afaf3cf"} Jul 06 23:49:34.900720 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:34.900663 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cj74h" podStartSLOduration=7.043033648 podStartE2EDuration="40.900646924s" totalImagesPullingTime="33.857613276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:34.8994146 +0000 UTC m=+41.829472126" watchObservedRunningTime="2026-07-06 23:49:34.900646924 +0000 UTC m=+41.830704442" Jul 06 23:49:36.203563 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.203517 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:36.203948 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:36.203664 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:36.203948 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:36.203729 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.203713516 +0000 UTC m=+51.133770998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:36.885206 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.885110 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" event={"ID":"b60ae206-9e76-4030-8791-5e6975a48b7d","Type":"ContainerStarted","Data":"617f55ded2eda582c2eeaa9a2cf6f1a690b4288865f5afcd29efb5492b8e4f8b"} Jul 06 23:49:36.886791 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.886740 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" event={"ID":"983d417b-c423-4a1f-b287-bfc925dcf1c0","Type":"ContainerStarted","Data":"143797f6abd8cc9faea5b572710481e2490214733d2a6f8ec910362547b50e8c"} Jul 06 23:49:36.888393 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.888374 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/0.log" Jul 06 23:49:36.888555 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.888409 2556 generic.go:356] "Generic (PLEG): container finished" podID="cd94558a-fa48-47f1-b582-d14568f0d0a4" containerID="4ad60bbafcef18dac11748aee43886555be97bbaa7ed285478c428f32b87ac8b" exitCode=255 Jul 06 23:49:36.888555 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.888437 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" event={"ID":"cd94558a-fa48-47f1-b582-d14568f0d0a4","Type":"ContainerDied","Data":"4ad60bbafcef18dac11748aee43886555be97bbaa7ed285478c428f32b87ac8b"} Jul 06 23:49:36.888683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.888666 2556 scope.go:117] "RemoveContainer" containerID="4ad60bbafcef18dac11748aee43886555be97bbaa7ed285478c428f32b87ac8b" Jul 06 23:49:36.890063 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.890043 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" event={"ID":"0dca4762-89a1-4978-aef9-09093526d95e","Type":"ContainerStarted","Data":"fdab9af2718fb6ba163022fad8025bb8ac58d94d123ee28f7349cec46e997ef1"} Jul 06 23:49:36.907029 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.906937 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" podStartSLOduration=2.683303562 podStartE2EDuration="6.906923604s" totalImagesPullingTime="4.223620042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:36.90657765 +0000 UTC m=+43.836635156" watchObservedRunningTime="2026-07-06 23:49:36.906923604 +0000 UTC m=+43.836981110" Jul 06 23:49:36.926635 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.926482 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-77dm4" podStartSLOduration=2.715846499 podStartE2EDuration="6.92646499s" totalImagesPullingTime="4.210618491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:36.925570752 +0000 UTC m=+43.855628259" watchObservedRunningTime="2026-07-06 23:49:36.92646499 +0000 UTC m=+43.856522495" Jul 06 23:49:36.972528 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:36.972483 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" podStartSLOduration=2.743128517 podStartE2EDuration="6.972470543s" totalImagesPullingTime="4.229342026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:36.971927669 +0000 UTC m=+43.901985174" watchObservedRunningTime="2026-07-06 23:49:36.972470543 +0000 UTC m=+43.902528274" Jul 06 23:49:37.895825 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.895797 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:49:37.896232 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.896217 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/0.log" Jul 06 23:49:37.896282 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.896249 2556 generic.go:356] "Generic (PLEG): container finished" podID="cd94558a-fa48-47f1-b582-d14568f0d0a4" containerID="974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e" exitCode=255 Jul 06 23:49:37.896396 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.896362 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" event={"ID":"cd94558a-fa48-47f1-b582-d14568f0d0a4","Type":"ContainerDied","Data":"974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e"} Jul 06 23:49:37.896526 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.896445 2556 scope.go:117] "RemoveContainer" containerID="4ad60bbafcef18dac11748aee43886555be97bbaa7ed285478c428f32b87ac8b" Jul 06 23:49:37.896600 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.896529 2556 scope.go:117] "RemoveContainer" containerID="974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e" Jul 06 23:49:37.896769 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:37.896721 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kjm87_openshift-console-operator(cd94558a-fa48-47f1-b582-d14568f0d0a4)\"" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" podUID="cd94558a-fa48-47f1-b582-d14568f0d0a4" Jul 06 23:49:37.974942 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:37.974908 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wlrkh"] Jul 06 23:49:38.002846 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.002821 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wlrkh"] Jul 06 23:49:38.002982 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.002943 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.005624 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.005607 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbs9b\"" Jul 06 23:49:38.005803 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.005787 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:49:38.005862 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.005853 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:49:38.117718 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.117679 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.117924 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.117731 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8qgk\" (UniqueName: \"kubernetes.io/projected/bab54804-6342-4511-bc8a-281eba5cd430-kube-api-access-r8qgk\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.117924 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.117835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.117924 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.117874 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bab54804-6342-4511-bc8a-281eba5cd430-crio-socket\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.118071 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.117963 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bab54804-6342-4511-bc8a-281eba5cd430-data-volume\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219293 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219211 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bab54804-6342-4511-bc8a-281eba5cd430-data-volume\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219293 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219273 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219487 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219303 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8qgk\" (UniqueName: \"kubernetes.io/projected/bab54804-6342-4511-bc8a-281eba5cd430-kube-api-access-r8qgk\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219487 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219331 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219487 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219347 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bab54804-6342-4511-bc8a-281eba5cd430-crio-socket\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219487 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.219435 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:38.219487 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.219448 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.219706 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.219518 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.71949889 +0000 UTC m=+45.649556386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : configmap "kube-rbac-proxy" not found Jul 06 23:49:38.219706 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.219533 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.719526545 +0000 UTC m=+45.649584027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.219706 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219543 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bab54804-6342-4511-bc8a-281eba5cd430-crio-socket\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.219706 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.219606 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bab54804-6342-4511-bc8a-281eba5cd430-data-volume\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.230014 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.229989 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8qgk\" (UniqueName: \"kubernetes.io/projected/bab54804-6342-4511-bc8a-281eba5cd430-kube-api-access-r8qgk\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.724643 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.724600 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.724660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.724723 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724764 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724814 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724839 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:46.724823718 +0000 UTC m=+53.654881220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : secret "router-metrics-certs-default" not found Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724850 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.724883 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724879 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:39.724866446 +0000 UTC m=+46.654923931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : configmap "kube-rbac-proxy" not found Jul 06 23:49:38.725159 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.724904 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:39.724890687 +0000 UTC m=+46.654948188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.725159 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.724946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:38.725159 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.725089 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:46.725073712 +0000 UTC m=+53.655131198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:38.901379 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.901347 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:49:38.901790 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:38.901676 2556 scope.go:117] "RemoveContainer" containerID="974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e" Jul 06 23:49:38.901868 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:38.901848 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kjm87_openshift-console-operator(cd94558a-fa48-47f1-b582-d14568f0d0a4)\"" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" podUID="cd94558a-fa48-47f1-b582-d14568f0d0a4" Jul 06 23:49:39.732551 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:39.732514 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:39.732838 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:39.732576 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:39.732838 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:39.732629 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:39.732838 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:39.732688 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:41.732672138 +0000 UTC m=+48.662729623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : configmap "kube-rbac-proxy" not found Jul 06 23:49:39.732838 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:39.732734 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:39.732838 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:39.732812 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:41.73279979 +0000 UTC m=+48.662857272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:40.755562 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:40.755533 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpbff_47916393-3c42-4fff-ac7e-3c1a27df4df1/dns-node-resolver/0.log" Jul 06 23:49:41.172743 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.172661 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:41.172743 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.172718 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:41.173110 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.173095 2556 scope.go:117] "RemoveContainer" containerID="974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e" Jul 06 23:49:41.173264 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:41.173245 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kjm87_openshift-console-operator(cd94558a-fa48-47f1-b582-d14568f0d0a4)\"" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" podUID="cd94558a-fa48-47f1-b582-d14568f0d0a4" Jul 06 23:49:41.554713 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.554686 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6nbc_1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38/node-ca/0.log" Jul 06 23:49:41.748096 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.748054 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:41.748256 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:41.748119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:41.748256 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:41.748169 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:41.748256 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:41.748253 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.748230879 +0000 UTC m=+52.678288368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : configmap "kube-rbac-proxy" not found Jul 06 23:49:41.748368 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:41.748265 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:41.748368 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:41.748304 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.748293472 +0000 UTC m=+52.678350971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:42.556457 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:42.556427 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wmgj8_2392b3d1-2fbc-4704-9f40-60f4d730adf7/serve-healthcheck-canary/0.log" Jul 06 23:49:43.160372 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:43.160340 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-w5vzz_b60ae206-9e76-4030-8791-5e6975a48b7d/kube-storage-version-migrator-operator/0.log" Jul 06 23:49:44.269294 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:44.269260 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:49:44.269671 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:44.269406 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:44.269671 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:44.269469 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls podName:73de84aa-74d2-4217-9e31-d069995b0a3a nodeName:}" failed. No retries permitted until 2026-07-06 23:50:00.269453913 +0000 UTC m=+67.199511395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls") pod "dns-default-mn5vl" (UID: "73de84aa-74d2-4217-9e31-d069995b0a3a") : secret "dns-default-metrics-tls" not found Jul 06 23:49:45.783650 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:45.783613 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:45.784040 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:45.783672 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:45.784040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:45.783777 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:45.784040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:45.783854 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:45.784040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:45.783857 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.783839576 +0000 UTC m=+60.713897062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : configmap "kube-rbac-proxy" not found Jul 06 23:49:45.784040 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:45.783924 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls podName:bab54804-6342-4511-bc8a-281eba5cd430 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.783907549 +0000 UTC m=+60.713965045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wlrkh" (UID: "bab54804-6342-4511-bc8a-281eba5cd430") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:46.793110 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:46.793074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:46.793539 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:46.793126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:46.793539 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:49:46.793237 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle podName:c1bfd36e-da1a-49d8-859e-1c66d4bc3e98 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:02.793220919 +0000 UTC m=+69.723278419 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle") pod "router-default-9b6988657-sb8wz" (UID: "c1bfd36e-da1a-49d8-859e-1c66d4bc3e98") : configmap references non-existent config key: service-ca.crt Jul 06 23:49:46.795381 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:46.795358 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-metrics-certs\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:49:53.838388 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.838360 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4b9rn" Jul 06 23:49:53.846940 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.846917 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:53.847105 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.846962 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:53.847424 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.847402 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bab54804-6342-4511-bc8a-281eba5cd430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:53.848963 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.848945 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bab54804-6342-4511-bc8a-281eba5cd430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wlrkh\" (UID: \"bab54804-6342-4511-bc8a-281eba5cd430\") " pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:53.914003 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.913977 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbs9b\"" Jul 06 23:49:53.921494 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:53.921479 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wlrkh" Jul 06 23:49:54.062344 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:54.062310 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wlrkh"] Jul 06 23:49:54.065198 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:54.065166 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab54804_6342_4511_bc8a_281eba5cd430.slice/crio-80cf674e29fd9324b45bf436e985ec37fb4fa1d561173d99249a69fe0e95da45 WatchSource:0}: Error finding container 80cf674e29fd9324b45bf436e985ec37fb4fa1d561173d99249a69fe0e95da45: Status 404 returned error can't find the container with id 80cf674e29fd9324b45bf436e985ec37fb4fa1d561173d99249a69fe0e95da45 Jul 06 23:49:54.952441 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:54.952257 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wlrkh" event={"ID":"bab54804-6342-4511-bc8a-281eba5cd430","Type":"ContainerStarted","Data":"80cf674e29fd9324b45bf436e985ec37fb4fa1d561173d99249a69fe0e95da45"} Jul 06 23:49:56.584461 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:56.584433 2556 scope.go:117] "RemoveContainer" containerID="974ab8733e198c5a80febe17546417927e03475015c0cfb2c3c1ca53616d929e" Jul 06 23:49:57.809712 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.809674 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj"] Jul 06 23:49:57.811860 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.811833 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.815564 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.815541 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 23:49:57.815685 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.815542 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5szbw\"" Jul 06 23:49:57.815685 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.815544 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 23:49:57.825489 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.825468 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9"] Jul 06 23:49:57.827669 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.827652 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.831218 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.831110 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4tmkz\"" Jul 06 23:49:57.832374 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.832183 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 23:49:57.832374 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.832233 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 23:49:57.832374 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.832239 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 23:49:57.832950 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.832929 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj"] Jul 06 23:49:57.833708 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.833690 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 23:49:57.852235 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.852208 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9"] Jul 06 23:49:57.884737 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.884658 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd02b66-6506-48e3-b3d6-5738cbab656e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.884911 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.884731 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r4x5\" (UniqueName: \"kubernetes.io/projected/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-kube-api-access-7r4x5\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.884911 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.884872 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.885031 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.884922 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd02b66-6506-48e3-b3d6-5738cbab656e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.921261 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.921216 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn"] Jul 06 23:49:57.923480 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.923448 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c847bf6bd-k62x9"] Jul 06 23:49:57.923609 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.923591 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" Jul 06 23:49:57.925430 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.925405 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.927072 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.927050 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-69rj4\"" Jul 06 23:49:57.929863 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.929842 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:49:57.930794 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.930769 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:49:57.930947 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.930928 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-csfd5\"" Jul 06 23:49:57.933410 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.933390 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:49:57.935110 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.935060 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:49:57.952494 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.952429 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn"] Jul 06 23:49:57.953619 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.953598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c847bf6bd-k62x9"] Jul 06 23:49:57.986353 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986326 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r4x5\" (UniqueName: \"kubernetes.io/projected/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-kube-api-access-7r4x5\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.986475 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986408 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-certificates\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.986475 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986441 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-trusted-ca\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.986578 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986562 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-image-registry-private-configuration\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.986635 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986600 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.986682 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-ca-trust-extracted\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986721 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-installation-pull-secrets\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986830 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd02b66-6506-48e3-b3d6-5738cbab656e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986882 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhl7\" (UniqueName: \"kubernetes.io/projected/3d4f12b6-9a13-47de-8c1b-252de923e5db-kube-api-access-8dhl7\") pod \"network-check-source-989f8b8dc-8kfsn\" (UID: \"3d4f12b6-9a13-47de-8c1b-252de923e5db\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986945 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd02b66-6506-48e3-b3d6-5738cbab656e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.986978 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-bound-sa-token\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.987030 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-tls\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.987055 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxxt\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-kube-api-access-2wxxt\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:57.988842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.988393 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd02b66-6506-48e3-b3d6-5738cbab656e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:57.990035 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.990016 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:57.990693 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:57.990675 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd02b66-6506-48e3-b3d6-5738cbab656e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ktkbj\" (UID: \"3dd02b66-6506-48e3-b3d6-5738cbab656e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:58.006686 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.006666 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r4x5\" (UniqueName: \"kubernetes.io/projected/9b702f5a-4083-4eb5-875e-2fae8d80e4ba-kube-api-access-7r4x5\") pod \"managed-serviceaccount-addon-agent-796c9d56c6-df4b9\" (UID: \"9b702f5a-4083-4eb5-875e-2fae8d80e4ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:58.088255 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-bound-sa-token\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088368 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088277 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-tls\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088368 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088298 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxxt\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-kube-api-access-2wxxt\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088368 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088330 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-certificates\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088368 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-trusted-ca\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088536 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088417 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-image-registry-private-configuration\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088536 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088456 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-ca-trust-extracted\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088536 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088483 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-installation-pull-secrets\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.088671 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088652 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhl7\" (UniqueName: \"kubernetes.io/projected/3d4f12b6-9a13-47de-8c1b-252de923e5db-kube-api-access-8dhl7\") pod \"network-check-source-989f8b8dc-8kfsn\" (UID: \"3d4f12b6-9a13-47de-8c1b-252de923e5db\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" Jul 06 23:49:58.089261 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.088927 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-ca-trust-extracted\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.089516 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.089492 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-certificates\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.089629 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.089598 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-trusted-ca\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.091090 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.091055 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-installation-pull-secrets\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.091183 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.091149 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-image-registry-private-configuration\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.091183 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.091174 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-registry-tls\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.104182 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.104160 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:58.104284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.104267 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxxt\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-kube-api-access-2wxxt\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.104502 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.104483 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7114e426-2c4b-4bbf-99f1-3ae91eed8f07-bound-sa-token\") pod \"image-registry-6c847bf6bd-k62x9\" (UID: \"7114e426-2c4b-4bbf-99f1-3ae91eed8f07\") " pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.115285 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.115266 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:58.122593 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.122571 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" Jul 06 23:49:58.125651 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.125634 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhl7\" (UniqueName: \"kubernetes.io/projected/3d4f12b6-9a13-47de-8c1b-252de923e5db-kube-api-access-8dhl7\") pod \"network-check-source-989f8b8dc-8kfsn\" (UID: \"3d4f12b6-9a13-47de-8c1b-252de923e5db\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" Jul 06 23:49:58.145586 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.145537 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" Jul 06 23:49:58.236548 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.236247 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" Jul 06 23:49:58.242985 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.242961 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:58.260858 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.260774 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj"] Jul 06 23:49:58.265494 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.265456 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd02b66_6506_48e3_b3d6_5738cbab656e.slice/crio-5282076d529a363f3ecf8e4b45ab041df5cb1eb9bb4366541cbc9efa75c00929 WatchSource:0}: Error finding container 5282076d529a363f3ecf8e4b45ab041df5cb1eb9bb4366541cbc9efa75c00929: Status 404 returned error can't find the container with id 5282076d529a363f3ecf8e4b45ab041df5cb1eb9bb4366541cbc9efa75c00929 Jul 06 23:49:58.293258 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.293230 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9"] Jul 06 23:49:58.298899 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.297612 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b702f5a_4083_4eb5_875e_2fae8d80e4ba.slice/crio-a7c7ce571a6f9cf77e7f1aded072703922bff742f5551c0772d9b769fae57887 WatchSource:0}: Error finding container a7c7ce571a6f9cf77e7f1aded072703922bff742f5551c0772d9b769fae57887: Status 404 returned error can't find the container with id a7c7ce571a6f9cf77e7f1aded072703922bff742f5551c0772d9b769fae57887 Jul 06 23:49:58.392331 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.392301 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn"] Jul 06 23:49:58.392662 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.392629 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:58.392806 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.392682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:58.392806 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.392730 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:58.394913 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.394880 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4f12b6_9a13_47de_8c1b_252de923e5db.slice/crio-78c0fe334852be7c8d822070a5e9d8d63d4023071b2a07b1fce755fcb583f4e2 WatchSource:0}: Error finding container 78c0fe334852be7c8d822070a5e9d8d63d4023071b2a07b1fce755fcb583f4e2: Status 404 returned error can't find the container with id 78c0fe334852be7c8d822070a5e9d8d63d4023071b2a07b1fce755fcb583f4e2 Jul 06 23:49:58.395479 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.395458 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6hx6\" (UniqueName: \"kubernetes.io/projected/4312a9f3-c7a4-4382-bcae-ff87dd8379ef-kube-api-access-p6hx6\") pod \"network-check-target-hhnlw\" (UID: \"4312a9f3-c7a4-4382-bcae-ff87dd8379ef\") " pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:58.396115 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.396095 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:58.396299 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.396282 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:58.400741 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.400723 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dwstx\"" Jul 06 23:49:58.406228 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.406191 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/283f81b9-52b3-482e-9f4b-6bf4cf1c09f2-original-pull-secret\") pod \"global-pull-secret-syncer-ckczt\" (UID: \"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2\") " pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:58.406386 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.406365 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21436535-c81e-4688-bc63-5c0720cba1cd-metrics-certs\") pod \"network-metrics-daemon-7xjzp\" (UID: \"21436535-c81e-4688-bc63-5c0720cba1cd\") " pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:58.408344 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.408322 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:49:58.422969 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.422943 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c847bf6bd-k62x9"] Jul 06 23:49:58.500666 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.500634 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7114e426_2c4b_4bbf_99f1_3ae91eed8f07.slice/crio-d752d544fa904f3ed312cfcd41923860829f1c24a20943a77ab38db3a52515cf WatchSource:0}: Error finding container d752d544fa904f3ed312cfcd41923860829f1c24a20943a77ab38db3a52515cf: Status 404 returned error can't find the container with id d752d544fa904f3ed312cfcd41923860829f1c24a20943a77ab38db3a52515cf Jul 06 23:49:58.593812 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.593788 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckczt" Jul 06 23:49:58.635882 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.635815 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hhnlw"] Jul 06 23:49:58.640122 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.640091 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4312a9f3_c7a4_4382_bcae_ff87dd8379ef.slice/crio-44e7ace0e79878083c1c64370cf7f53a5e490eb828655603ad6a188109a9c7d8 WatchSource:0}: Error finding container 44e7ace0e79878083c1c64370cf7f53a5e490eb828655603ad6a188109a9c7d8: Status 404 returned error can't find the container with id 44e7ace0e79878083c1c64370cf7f53a5e490eb828655603ad6a188109a9c7d8 Jul 06 23:49:58.707240 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.707210 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzg9j\"" Jul 06 23:49:58.714418 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.714394 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xjzp" Jul 06 23:49:58.732866 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.732813 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckczt"] Jul 06 23:49:58.736427 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:49:58.736390 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283f81b9_52b3_482e_9f4b_6bf4cf1c09f2.slice/crio-3e33390cf5ff683c6fe93f9aa30adfec939ccc20f116d50b7351b6039291249e WatchSource:0}: Error finding container 3e33390cf5ff683c6fe93f9aa30adfec939ccc20f116d50b7351b6039291249e: Status 404 returned error can't find the container with id 3e33390cf5ff683c6fe93f9aa30adfec939ccc20f116d50b7351b6039291249e Jul 06 23:49:58.879247 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.879147 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xjzp"] Jul 06 23:49:58.970484 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.970227 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" event={"ID":"9b702f5a-4083-4eb5-875e-2fae8d80e4ba","Type":"ContainerStarted","Data":"a7c7ce571a6f9cf77e7f1aded072703922bff742f5551c0772d9b769fae57887"} Jul 06 23:49:58.979398 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.979243 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wlrkh" event={"ID":"bab54804-6342-4511-bc8a-281eba5cd430","Type":"ContainerStarted","Data":"476365211aeefcae2dc5667c2d240f45676b72b8191893ea6f96428d1920a4f2"} Jul 06 23:49:58.979398 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.979344 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wlrkh" event={"ID":"bab54804-6342-4511-bc8a-281eba5cd430","Type":"ContainerStarted","Data":"02e3ac5e77824e1bb4d9cdac91542603adf6242c04e55fde031c2549bd005abb"} Jul 06 23:49:58.985119 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.985033 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xjzp" event={"ID":"21436535-c81e-4688-bc63-5c0720cba1cd","Type":"ContainerStarted","Data":"a89b5d41e0750d303b247ee9cc0af7b3f8afa4966f48b6dac6baa485432a20a4"} Jul 06 23:49:58.989316 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.989250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" event={"ID":"3dd02b66-6506-48e3-b3d6-5738cbab656e","Type":"ContainerStarted","Data":"5282076d529a363f3ecf8e4b45ab041df5cb1eb9bb4366541cbc9efa75c00929"} Jul 06 23:49:58.995890 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.995523 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:49:58.995890 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.995602 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" event={"ID":"cd94558a-fa48-47f1-b582-d14568f0d0a4","Type":"ContainerStarted","Data":"d06dd4c588162146160a4a91b29e40cd5aece3925e9827aae3ddf3200f6497b2"} Jul 06 23:49:58.996295 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:58.996245 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:49:59.001113 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.001036 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" event={"ID":"7114e426-2c4b-4bbf-99f1-3ae91eed8f07","Type":"ContainerStarted","Data":"4ce161629251fe5df1878e99b837fc5e283f08236338ac9a63554629d60c8b6c"} Jul 06 23:49:59.001113 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.001081 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" event={"ID":"7114e426-2c4b-4bbf-99f1-3ae91eed8f07","Type":"ContainerStarted","Data":"d752d544fa904f3ed312cfcd41923860829f1c24a20943a77ab38db3a52515cf"} Jul 06 23:49:59.001345 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.001301 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:49:59.003694 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.003636 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" event={"ID":"3d4f12b6-9a13-47de-8c1b-252de923e5db","Type":"ContainerStarted","Data":"78c0fe334852be7c8d822070a5e9d8d63d4023071b2a07b1fce755fcb583f4e2"} Jul 06 23:49:59.005316 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.005231 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckczt" event={"ID":"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2","Type":"ContainerStarted","Data":"3e33390cf5ff683c6fe93f9aa30adfec939ccc20f116d50b7351b6039291249e"} Jul 06 23:49:59.006846 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.006798 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hhnlw" event={"ID":"4312a9f3-c7a4-4382-bcae-ff87dd8379ef","Type":"ContainerStarted","Data":"44e7ace0e79878083c1c64370cf7f53a5e490eb828655603ad6a188109a9c7d8"} Jul 06 23:49:59.016795 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.015977 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" podStartSLOduration=24.799962461 podStartE2EDuration="29.015959368s" totalImagesPullingTime="4.215996907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:59.015045109 +0000 UTC m=+65.945102614" watchObservedRunningTime="2026-07-06 23:49:59.015959368 +0000 UTC m=+65.946016873" Jul 06 23:49:59.035370 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.034430 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" podStartSLOduration=2.034411258 podStartE2EDuration="2.034411258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:59.033608022 +0000 UTC m=+65.963665527" watchObservedRunningTime="2026-07-06 23:49:59.034411258 +0000 UTC m=+65.964468810" Jul 06 23:49:59.356272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:49:59.356023 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-kjm87" Jul 06 23:50:00.317054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:00.316531 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:50:00.321936 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:00.321831 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73de84aa-74d2-4217-9e31-d069995b0a3a-metrics-tls\") pod \"dns-default-mn5vl\" (UID: \"73de84aa-74d2-4217-9e31-d069995b0a3a\") " pod="openshift-dns/dns-default-mn5vl" Jul 06 23:50:00.494170 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:00.493961 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k5jpj\"" Jul 06 23:50:00.501541 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:00.501315 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mn5vl" Jul 06 23:50:02.509860 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:02.509773 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mn5vl"] Jul 06 23:50:02.844030 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:02.843989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:02.844695 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:02.844673 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1bfd36e-da1a-49d8-859e-1c66d4bc3e98-service-ca-bundle\") pod \"router-default-9b6988657-sb8wz\" (UID: \"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98\") " pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:03.071157 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:03.071121 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-crbjf\"" Jul 06 23:50:03.078457 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:03.078429 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:06.148490 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:06.148455 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73de84aa_74d2_4217_9e31_d069995b0a3a.slice/crio-099e561e947e8faf4725f2c41b26403c8d7a67660c184a1be97080333eb7f2d2 WatchSource:0}: Error finding container 099e561e947e8faf4725f2c41b26403c8d7a67660c184a1be97080333eb7f2d2: Status 404 returned error can't find the container with id 099e561e947e8faf4725f2c41b26403c8d7a67660c184a1be97080333eb7f2d2 Jul 06 23:50:06.400514 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:06.399930 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9b6988657-sb8wz"] Jul 06 23:50:06.404894 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:06.404871 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1bfd36e_da1a_49d8_859e_1c66d4bc3e98.slice/crio-8b3f25fc5c2cece767b422e4ba5707be5d2120d9feba10c8f8ad727eb789b018 WatchSource:0}: Error finding container 8b3f25fc5c2cece767b422e4ba5707be5d2120d9feba10c8f8ad727eb789b018: Status 404 returned error can't find the container with id 8b3f25fc5c2cece767b422e4ba5707be5d2120d9feba10c8f8ad727eb789b018 Jul 06 23:50:07.047388 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.047348 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" event={"ID":"9b702f5a-4083-4eb5-875e-2fae8d80e4ba","Type":"ContainerStarted","Data":"0bc885b3b7d6f2e8f0c3652ac9c5b8948644b164740d9321686a0afed86b497c"} Jul 06 23:50:07.049820 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.049767 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wlrkh" event={"ID":"bab54804-6342-4511-bc8a-281eba5cd430","Type":"ContainerStarted","Data":"ec392b63e852ec3ae4f279d6a162cbe7b44ded06f83b1abafc145b0abfd440cd"} Jul 06 23:50:07.051782 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.051738 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xjzp" event={"ID":"21436535-c81e-4688-bc63-5c0720cba1cd","Type":"ContainerStarted","Data":"e62cc320840a649676c28ce39fc230972105ae40645d2bcd8cf5d8a22dd2fe81"} Jul 06 23:50:07.051893 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.051802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xjzp" event={"ID":"21436535-c81e-4688-bc63-5c0720cba1cd","Type":"ContainerStarted","Data":"bfd3e94c0361f05ec30140d53576717b583ed09c36f8c7e4b60397e7fd20bd16"} Jul 06 23:50:07.053453 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.053430 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" event={"ID":"3dd02b66-6506-48e3-b3d6-5738cbab656e","Type":"ContainerStarted","Data":"479ddb919bc31968382c0b7df70aee52979d9d1d740764e9af07c3cf2d75e3a9"} Jul 06 23:50:07.054973 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.054909 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9b6988657-sb8wz" event={"ID":"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98","Type":"ContainerStarted","Data":"49fa4bfadaa1ea0b7ca531d9071114747243ce4bde1c3805385773b0521e5eee"} Jul 06 23:50:07.054973 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.054947 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9b6988657-sb8wz" event={"ID":"c1bfd36e-da1a-49d8-859e-1c66d4bc3e98","Type":"ContainerStarted","Data":"8b3f25fc5c2cece767b422e4ba5707be5d2120d9feba10c8f8ad727eb789b018"} Jul 06 23:50:07.056460 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.056439 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" event={"ID":"3d4f12b6-9a13-47de-8c1b-252de923e5db","Type":"ContainerStarted","Data":"cdae896b6e3d5f820e6e8cba3c7544969b023fc2ff6e9505cf429a21ced2436a"} Jul 06 23:50:07.057710 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.057684 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn5vl" event={"ID":"73de84aa-74d2-4217-9e31-d069995b0a3a","Type":"ContainerStarted","Data":"099e561e947e8faf4725f2c41b26403c8d7a67660c184a1be97080333eb7f2d2"} Jul 06 23:50:07.059035 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.059015 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckczt" event={"ID":"283f81b9-52b3-482e-9f4b-6bf4cf1c09f2","Type":"ContainerStarted","Data":"8a2cdc181e5d7fc8ff7da62833095cd3023b097a435c25e0cf87027e7105b25f"} Jul 06 23:50:07.060595 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.060572 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hhnlw" event={"ID":"4312a9f3-c7a4-4382-bcae-ff87dd8379ef","Type":"ContainerStarted","Data":"70318ea4bc4ea60a922eb2d990cd360629a4fb7694d922d62d44708d86df27e7"} Jul 06 23:50:07.060766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.060626 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:50:07.064812 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.064768 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-796c9d56c6-df4b9" podStartSLOduration=2.136212656 podStartE2EDuration="10.064733187s" totalImagesPullingTime="7.928520531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.063885635 +0000 UTC m=+73.993943144" watchObservedRunningTime="2026-07-06 23:50:07.064733187 +0000 UTC m=+73.994790695" Jul 06 23:50:07.079148 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.079094 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:07.081129 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.080843 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hhnlw" podStartSLOduration=65.505040585 podStartE2EDuration="1m13.080820556s" totalImagesPullingTime="7.575779971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.079299516 +0000 UTC m=+74.009357033" watchObservedRunningTime="2026-07-06 23:50:07.080820556 +0000 UTC m=+74.010878054" Jul 06 23:50:07.082109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.082091 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:07.102501 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.102465 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9b6988657-sb8wz" podStartSLOduration=37.102453862 podStartE2EDuration="37.102453862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.101739363 +0000 UTC m=+74.031796867" watchObservedRunningTime="2026-07-06 23:50:07.102453862 +0000 UTC m=+74.032511367" Jul 06 23:50:07.119884 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.119842 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7xjzp" podStartSLOduration=65.809768092 podStartE2EDuration="1m13.119825027s" totalImagesPullingTime="7.310056935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.119018634 +0000 UTC m=+74.049076139" watchObservedRunningTime="2026-07-06 23:50:07.119825027 +0000 UTC m=+74.049882533" Jul 06 23:50:07.137739 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.137696 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ktkbj" podStartSLOduration=2.206729751 podStartE2EDuration="10.13768314s" totalImagesPullingTime="7.930953389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.136911968 +0000 UTC m=+74.066969473" watchObservedRunningTime="2026-07-06 23:50:07.13768314 +0000 UTC m=+74.067740623" Jul 06 23:50:07.152649 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.152591 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ckczt" podStartSLOduration=65.659456797 podStartE2EDuration="1m13.152576084s" totalImagesPullingTime="7.493119287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.152199597 +0000 UTC m=+74.082257102" watchObservedRunningTime="2026-07-06 23:50:07.152576084 +0000 UTC m=+74.082633589" Jul 06 23:50:07.175710 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.175627 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wlrkh" podStartSLOduration=18.123340681 podStartE2EDuration="30.175607278s" totalImagesPullingTime="12.052266597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.174707092 +0000 UTC m=+74.104764587" watchObservedRunningTime="2026-07-06 23:50:07.175607278 +0000 UTC m=+74.105664782" Jul 06 23:50:07.192139 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:07.192095 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kfsn" podStartSLOduration=2.343449866 podStartE2EDuration="10.192078751s" totalImagesPullingTime="7.848628885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.19173754 +0000 UTC m=+74.121795045" watchObservedRunningTime="2026-07-06 23:50:07.192078751 +0000 UTC m=+74.122136254" Jul 06 23:50:08.066488 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.066450 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn5vl" event={"ID":"73de84aa-74d2-4217-9e31-d069995b0a3a","Type":"ContainerStarted","Data":"63a2ae3517443644d601081add83e7c3ba2af441956498abfff869faa90b5970"} Jul 06 23:50:08.066694 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.066514 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mn5vl" event={"ID":"73de84aa-74d2-4217-9e31-d069995b0a3a","Type":"ContainerStarted","Data":"ed7c1f7c9f90ec6437443b012b358a922d1f74d4a58f49b0ed15471cee90df4c"} Jul 06 23:50:08.070774 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.068234 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mn5vl" Jul 06 23:50:08.070774 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.068286 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:08.072433 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.072411 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9b6988657-sb8wz" Jul 06 23:50:08.085709 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:08.085665 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mn5vl" podStartSLOduration=38.545944547 podStartE2EDuration="40.085649285s" totalImagesPullingTime="1.539704738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:08.084155137 +0000 UTC m=+75.014212642" watchObservedRunningTime="2026-07-06 23:50:08.085649285 +0000 UTC m=+75.015706788" Jul 06 23:50:14.980726 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.980695 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks"] Jul 06 23:50:14.983777 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.983760 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:14.987447 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987413 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 23:50:14.987447 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987431 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nv8vw\"" Jul 06 23:50:14.987447 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987438 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:50:14.987700 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987495 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 23:50:14.987700 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987570 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:50:14.987700 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.987611 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:50:14.994449 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.994426 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks"] Jul 06 23:50:14.998717 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:14.998696 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gx7vd"] Jul 06 23:50:15.001593 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.001578 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.006222 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.006204 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:50:15.006348 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.006270 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:50:15.006348 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.006297 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:50:15.006348 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.006319 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hhxhk\"" Jul 06 23:50:15.143132 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143006 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-sys\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143132 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143037 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8675w\" (UniqueName: \"kubernetes.io/projected/5e6dd046-0cd6-4758-9676-10a77836b25b-kube-api-access-8675w\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143132 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143062 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-textfile\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143132 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143117 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-wtmp\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143181 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143215 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143245 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-accelerators-collector-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143271 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e40c58d-524f-4a93-9947-8296c8e84cbc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143296 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j8sc\" (UniqueName: \"kubernetes.io/projected/4e40c58d-524f-4a93-9947-8296c8e84cbc-kube-api-access-2j8sc\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143322 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143366 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-metrics-client-ca\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143400 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-root\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.143464 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.143440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244764 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244715 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-sys\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8675w\" (UniqueName: \"kubernetes.io/projected/5e6dd046-0cd6-4758-9676-10a77836b25b-kube-api-access-8675w\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244812 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-textfile\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-wtmp\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244843 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-sys\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244867 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.244946 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244929 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:50:15.244960 2556 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.244971 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-accelerators-collector-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e40c58d-524f-4a93-9947-8296c8e84cbc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:50:15.245017 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls podName:4e40c58d-524f-4a93-9947-8296c8e84cbc nodeName:}" failed. No retries permitted until 2026-07-06 23:50:15.745000034 +0000 UTC m=+82.675057547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-x5gks" (UID: "4e40c58d-524f-4a93-9947-8296c8e84cbc") : secret "openshift-state-metrics-tls" not found Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245027 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-wtmp\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245041 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j8sc\" (UniqueName: \"kubernetes.io/projected/4e40c58d-524f-4a93-9947-8296c8e84cbc-kube-api-access-2j8sc\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245112 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-metrics-client-ca\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245140 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-root\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245164 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245272 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245202 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-textfile\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:50:15.245283 2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:50:15.245326 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls podName:5e6dd046-0cd6-4758-9676-10a77836b25b nodeName:}" failed. No retries permitted until 2026-07-06 23:50:15.745311413 +0000 UTC m=+82.675368923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls") pod "node-exporter-gx7vd" (UID: "5e6dd046-0cd6-4758-9676-10a77836b25b") : secret "node-exporter-tls" not found Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245376 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e6dd046-0cd6-4758-9676-10a77836b25b-root\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245570 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-accelerators-collector-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245662 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6dd046-0cd6-4758-9676-10a77836b25b-metrics-client-ca\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.245772 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.245732 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e40c58d-524f-4a93-9947-8296c8e84cbc-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.247490 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.247464 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.247569 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.247507 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.255140 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.255121 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8675w\" (UniqueName: \"kubernetes.io/projected/5e6dd046-0cd6-4758-9676-10a77836b25b-kube-api-access-8675w\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.259028 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.259011 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j8sc\" (UniqueName: \"kubernetes.io/projected/4e40c58d-524f-4a93-9947-8296c8e84cbc-kube-api-access-2j8sc\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.749476 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.749440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.749669 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.749519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.751695 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.751671 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e6dd046-0cd6-4758-9676-10a77836b25b-node-exporter-tls\") pod \"node-exporter-gx7vd\" (UID: \"5e6dd046-0cd6-4758-9676-10a77836b25b\") " pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.751842 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.751824 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e40c58d-524f-4a93-9947-8296c8e84cbc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x5gks\" (UID: \"4e40c58d-524f-4a93-9947-8296c8e84cbc\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.893443 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.893399 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" Jul 06 23:50:15.910958 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:15.910926 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gx7vd" Jul 06 23:50:15.921602 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:15.921575 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6dd046_0cd6_4758_9676_10a77836b25b.slice/crio-a96b398e37f2f2e9be0ba12b1e2d17fb25832aeb37f4940d969bbc12f7d551cf WatchSource:0}: Error finding container a96b398e37f2f2e9be0ba12b1e2d17fb25832aeb37f4940d969bbc12f7d551cf: Status 404 returned error can't find the container with id a96b398e37f2f2e9be0ba12b1e2d17fb25832aeb37f4940d969bbc12f7d551cf Jul 06 23:50:16.027188 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:16.027105 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks"] Jul 06 23:50:16.029907 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:16.029880 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e40c58d_524f_4a93_9947_8296c8e84cbc.slice/crio-3405fcd1b67a36fefdfe4c8a9bab307acdd9634a0be15d5d5f9549667a81d0d3 WatchSource:0}: Error finding container 3405fcd1b67a36fefdfe4c8a9bab307acdd9634a0be15d5d5f9549667a81d0d3: Status 404 returned error can't find the container with id 3405fcd1b67a36fefdfe4c8a9bab307acdd9634a0be15d5d5f9549667a81d0d3 Jul 06 23:50:16.096926 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:16.096893 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gx7vd" event={"ID":"5e6dd046-0cd6-4758-9676-10a77836b25b","Type":"ContainerStarted","Data":"a96b398e37f2f2e9be0ba12b1e2d17fb25832aeb37f4940d969bbc12f7d551cf"} Jul 06 23:50:16.100935 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:16.100856 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" event={"ID":"4e40c58d-524f-4a93-9947-8296c8e84cbc","Type":"ContainerStarted","Data":"3405fcd1b67a36fefdfe4c8a9bab307acdd9634a0be15d5d5f9549667a81d0d3"} Jul 06 23:50:17.106697 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.106672 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" event={"ID":"4e40c58d-524f-4a93-9947-8296c8e84cbc","Type":"ContainerStarted","Data":"43668ec1d78c5877f946ca653007e695e97931ef14cfe7d6c4b2944703290330"} Jul 06 23:50:17.106970 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.106728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" event={"ID":"4e40c58d-524f-4a93-9947-8296c8e84cbc","Type":"ContainerStarted","Data":"7cc9014789bc590f3a2b92d34521e4ba06418d8bf7a767d39615f07b7253d6a4"} Jul 06 23:50:17.929508 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.929474 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb"] Jul 06 23:50:17.932938 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.932923 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:17.935830 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935802 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935840 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e3l3drt0jbn23\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935871 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rvsl8\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935892 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935912 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935871 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 23:50:17.935964 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.935871 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 23:50:17.946219 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:17.946198 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb"] Jul 06 23:50:18.071452 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071423 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071617 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071617 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071542 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86462d59-0a9d-4f88-aa49-7bbf331b5386-metrics-client-ca\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071617 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071594 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071617 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071764 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071657 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc96z\" (UniqueName: \"kubernetes.io/projected/86462d59-0a9d-4f88-aa49-7bbf331b5386-kube-api-access-wc96z\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071764 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071696 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-grpc-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.071764 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.071715 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.111920 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.111889 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" event={"ID":"4e40c58d-524f-4a93-9947-8296c8e84cbc","Type":"ContainerStarted","Data":"488ebd03326672ee7c613e9c8c90034fd8549bb2e4c7367ebd1adf774f34ffe2"} Jul 06 23:50:18.113540 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.113513 2556 generic.go:356] "Generic (PLEG): container finished" podID="5e6dd046-0cd6-4758-9676-10a77836b25b" containerID="ad5f945c9ffe19207c6fe87111a56369c5b18093a6bfc3cc57234a4512354592" exitCode=0 Jul 06 23:50:18.113643 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.113557 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gx7vd" event={"ID":"5e6dd046-0cd6-4758-9676-10a77836b25b","Type":"ContainerDied","Data":"ad5f945c9ffe19207c6fe87111a56369c5b18093a6bfc3cc57234a4512354592"} Jul 06 23:50:18.133300 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.133262 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x5gks" podStartSLOduration=3.146457859 podStartE2EDuration="4.133247677s" totalImagesPullingTime="986.789818ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:18.131721594 +0000 UTC m=+85.061779106" watchObservedRunningTime="2026-07-06 23:50:18.133247677 +0000 UTC m=+85.063305181" Jul 06 23:50:18.172021 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172000 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172126 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172032 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172126 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172052 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc96z\" (UniqueName: \"kubernetes.io/projected/86462d59-0a9d-4f88-aa49-7bbf331b5386-kube-api-access-wc96z\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172126 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-grpc-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172126 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172126 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172358 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172136 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172358 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86462d59-0a9d-4f88-aa49-7bbf331b5386-metrics-client-ca\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.172927 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.172906 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86462d59-0a9d-4f88-aa49-7bbf331b5386-metrics-client-ca\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175067 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175034 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175163 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175133 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175163 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175152 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175271 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175195 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175271 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175254 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-grpc-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.175485 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.175468 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86462d59-0a9d-4f88-aa49-7bbf331b5386-secret-thanos-querier-tls\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.180970 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.180950 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc96z\" (UniqueName: \"kubernetes.io/projected/86462d59-0a9d-4f88-aa49-7bbf331b5386-kube-api-access-wc96z\") pod \"thanos-querier-7d8c99fb6f-dwxzb\" (UID: \"86462d59-0a9d-4f88-aa49-7bbf331b5386\") " pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.242182 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.242163 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:18.372465 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:18.372438 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb"] Jul 06 23:50:18.376012 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:18.375985 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86462d59_0a9d_4f88_aa49_7bbf331b5386.slice/crio-7d11d52670f7cca85ea1b793166e493beea13cd4302df0bfa1cc5eaf7f0f50da WatchSource:0}: Error finding container 7d11d52670f7cca85ea1b793166e493beea13cd4302df0bfa1cc5eaf7f0f50da: Status 404 returned error can't find the container with id 7d11d52670f7cca85ea1b793166e493beea13cd4302df0bfa1cc5eaf7f0f50da Jul 06 23:50:19.076487 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:19.076452 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mn5vl" Jul 06 23:50:19.120948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:19.120835 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gx7vd" event={"ID":"5e6dd046-0cd6-4758-9676-10a77836b25b","Type":"ContainerStarted","Data":"265f63f84fd44c5a53b43e9fc15dd311f1950ac5fd035a398074baf02033940f"} Jul 06 23:50:19.120948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:19.120911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gx7vd" event={"ID":"5e6dd046-0cd6-4758-9676-10a77836b25b","Type":"ContainerStarted","Data":"8eb4c1b239d3570e1420ff5ba22680bafaf7772a39d96ac1924ba6e89f74c17e"} Jul 06 23:50:19.122610 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:19.122558 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"7d11d52670f7cca85ea1b793166e493beea13cd4302df0bfa1cc5eaf7f0f50da"} Jul 06 23:50:19.160801 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:19.160737 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gx7vd" podStartSLOduration=3.91223277 podStartE2EDuration="5.160721036s" totalImagesPullingTime="1.182373006s" totalInitContainerRuntime="66.11526ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:19.159134757 +0000 UTC m=+86.089192262" watchObservedRunningTime="2026-07-06 23:50:19.160721036 +0000 UTC m=+86.090778556" Jul 06 23:50:20.022480 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:20.022445 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c847bf6bd-k62x9" Jul 06 23:50:21.133416 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.133375 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"d4a5900ad1c1e4bff0fc7a6372c2cabd66fc076f611c64da61748e21711efe1e"} Jul 06 23:50:21.133803 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.133451 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"fc125a1fbf5057819ed8533c771cd980b4fc215a575401eb9a52598e2a6260ee"} Jul 06 23:50:21.133803 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.133482 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"8d5cd13682a003ec1a7ad4c335f20d68e58fbd9a057a8da87fd24cc2b4a6b704"} Jul 06 23:50:21.184732 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.184696 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:50:21.188229 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.188212 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.190923 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.190869 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 23:50:21.191127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.190978 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 23:50:21.191214 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.191135 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 23:50:21.191214 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.191196 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-98wl7\"" Jul 06 23:50:21.191349 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.191210 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 23:50:21.192018 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.191993 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 23:50:21.192134 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192031 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 23:50:21.192134 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192082 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 23:50:21.192237 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192134 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 23:50:21.192237 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192138 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 23:50:21.192237 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192161 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 23:50:21.192388 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192260 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 23:50:21.192501 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192485 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 23:50:21.192541 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192505 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rv251l0o3mdl\"" Jul 06 23:50:21.192581 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192565 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 23:50:21.192581 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.192570 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 23:50:21.195423 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.195401 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 23:50:21.209233 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.209205 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.298900 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.298954 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.298984 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299026 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299065 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299101 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299133 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299187 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299219 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299259 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299301 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299338 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.300766 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299415 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299453 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299502 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299556 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc2m5\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299592 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299632 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.301389 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.299664 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400387 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400365 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400500 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400397 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400500 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400421 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400500 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400458 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400500 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400482 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400520 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400562 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc2m5\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400593 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400675 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.400705 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400703 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400730 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400789 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400814 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400863 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400949 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.400983 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401497 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.401339 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.401653 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.401627 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.402258 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.402233 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.402429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.402262 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.403568 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.403435 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.404678 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.404444 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.405408 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.405165 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407009 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407102 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407174 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407188 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407430 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407442 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407851 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407575 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.407851 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.407769 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.408127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.408067 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.408362 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.408312 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.408442 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.408373 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.409296 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.409273 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.409943 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.409925 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc2m5\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5\") pod \"prometheus-k8s-0\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.498470 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.498436 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:21.647045 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:21.646970 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:50:21.649980 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:50:21.649948 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ea3bd1_90cc_4a17_890f_b61649e20482.slice/crio-e0d3dd3b76835c9e4a5b383e70cc0a3eb24a365e231f893d35d7896b3de509ca WatchSource:0}: Error finding container e0d3dd3b76835c9e4a5b383e70cc0a3eb24a365e231f893d35d7896b3de509ca: Status 404 returned error can't find the container with id e0d3dd3b76835c9e4a5b383e70cc0a3eb24a365e231f893d35d7896b3de509ca Jul 06 23:50:22.140398 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.140359 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"9be7c6f1a0cd02461f08fa08bdfb9192e824ed8753ec4a9c9436e59cb82aaa6e"} Jul 06 23:50:22.140891 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.140429 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"3e6b9667e6b2638ca49c11813dcb6a10c595441ea0fb5acb4423f8fa48de82a1"} Jul 06 23:50:22.140891 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.140456 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" event={"ID":"86462d59-0a9d-4f88-aa49-7bbf331b5386","Type":"ContainerStarted","Data":"a9eb1f37e1eb95682e10c55811802d4fb7165a2792ed8c6b3bfff55817a9d111"} Jul 06 23:50:22.140891 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.140580 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:22.141639 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.141618 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"e0d3dd3b76835c9e4a5b383e70cc0a3eb24a365e231f893d35d7896b3de509ca"} Jul 06 23:50:22.168328 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:22.168283 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" podStartSLOduration=2.501589518 podStartE2EDuration="5.168270455s" totalImagesPullingTime="2.666680937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:22.166861523 +0000 UTC m=+89.096919033" watchObservedRunningTime="2026-07-06 23:50:22.168270455 +0000 UTC m=+89.098327959" Jul 06 23:50:23.147396 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:23.147308 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" exitCode=0 Jul 06 23:50:23.147884 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:23.147391 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} Jul 06 23:50:26.165482 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:26.165451 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} Jul 06 23:50:27.172796 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.172728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} Jul 06 23:50:27.173187 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.172853 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} Jul 06 23:50:27.173187 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.172887 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} Jul 06 23:50:27.173187 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.172914 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} Jul 06 23:50:27.173187 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.172939 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerStarted","Data":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} Jul 06 23:50:27.204858 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:27.204774 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.111533833 podStartE2EDuration="6.204739622s" totalImagesPullingTime="4.074601712s" totalInitContainerRuntime="18.604077ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:27.20311559 +0000 UTC m=+94.133173095" watchObservedRunningTime="2026-07-06 23:50:27.204739622 +0000 UTC m=+94.134797126" Jul 06 23:50:28.153847 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:28.153820 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d8c99fb6f-dwxzb" Jul 06 23:50:31.498879 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:31.498844 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:50:38.070473 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:38.070442 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hhnlw" Jul 06 23:50:52.275891 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:52.275860 2556 generic.go:356] "Generic (PLEG): container finished" podID="0dca4762-89a1-4978-aef9-09093526d95e" containerID="fdab9af2718fb6ba163022fad8025bb8ac58d94d123ee28f7349cec46e997ef1" exitCode=0 Jul 06 23:50:52.276224 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:52.275922 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" event={"ID":"0dca4762-89a1-4978-aef9-09093526d95e","Type":"ContainerDied","Data":"fdab9af2718fb6ba163022fad8025bb8ac58d94d123ee28f7349cec46e997ef1"} Jul 06 23:50:52.276224 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:52.276215 2556 scope.go:117] "RemoveContainer" containerID="fdab9af2718fb6ba163022fad8025bb8ac58d94d123ee28f7349cec46e997ef1" Jul 06 23:50:53.281877 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:53.281845 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-j68w9" event={"ID":"0dca4762-89a1-4978-aef9-09093526d95e","Type":"ContainerStarted","Data":"2d24d9e318f8ca4d9dfc46095424a768433ca767560f9ae1463da74b436781fc"} Jul 06 23:50:53.283239 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:53.283214 2556 generic.go:356] "Generic (PLEG): container finished" podID="b60ae206-9e76-4030-8791-5e6975a48b7d" containerID="617f55ded2eda582c2eeaa9a2cf6f1a690b4288865f5afcd29efb5492b8e4f8b" exitCode=0 Jul 06 23:50:53.283344 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:53.283289 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" event={"ID":"b60ae206-9e76-4030-8791-5e6975a48b7d","Type":"ContainerDied","Data":"617f55ded2eda582c2eeaa9a2cf6f1a690b4288865f5afcd29efb5492b8e4f8b"} Jul 06 23:50:53.283563 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:53.283550 2556 scope.go:117] "RemoveContainer" containerID="617f55ded2eda582c2eeaa9a2cf6f1a690b4288865f5afcd29efb5492b8e4f8b" Jul 06 23:50:54.289416 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:50:54.289382 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w5vzz" event={"ID":"b60ae206-9e76-4030-8791-5e6975a48b7d","Type":"ContainerStarted","Data":"d8bb23234dd606cabcf50900ca7a16c6b9bfee427b61e4f3e560c7c813d76f5a"} Jul 06 23:51:21.499240 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:21.499201 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:21.514925 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:21.514900 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:22.419584 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:22.419559 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:39.518213 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518158 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:39.518627 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518466 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="prometheus" containerID="cri-o://e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" gracePeriod=600 Jul 06 23:51:39.518627 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518479 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy" containerID="cri-o://342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" gracePeriod=600 Jul 06 23:51:39.518627 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518515 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-thanos" containerID="cri-o://dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" gracePeriod=600 Jul 06 23:51:39.518627 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518530 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="thanos-sidecar" containerID="cri-o://6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" gracePeriod=600 Jul 06 23:51:39.518627 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518551 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="config-reloader" containerID="cri-o://eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" gracePeriod=600 Jul 06 23:51:39.518931 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.518538 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-web" containerID="cri-o://4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" gracePeriod=600 Jul 06 23:51:39.756554 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.756529 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:39.867312 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867218 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867312 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867265 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867312 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867299 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867582 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867406 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867582 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867449 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867577 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867613 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867652 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867683 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867681 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867712 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867736 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867799 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867839 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867863 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.867919 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867901 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.868249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867926 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.868249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867963 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.868249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867988 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc2m5\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.868249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.868026 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.868249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.868065 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config\") pod \"c6ea3bd1-90cc-4a17-890f-b61649e20482\" (UID: \"c6ea3bd1-90cc-4a17-890f-b61649e20482\") " Jul 06 23:51:39.869257 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.867733 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869257 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.868705 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869257 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.868772 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869257 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.869188 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869590 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.869276 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869590 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.869406 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.869725 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.869639 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:39.870313 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.870286 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.870313 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.870299 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:51:39.870455 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.870426 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:51:39.870609 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.870579 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.871276 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.871250 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.871778 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.871718 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.871869 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.871779 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.872359 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.872332 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out" (OuterVolumeSpecName: "config-out") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:51:39.872478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.872436 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config" (OuterVolumeSpecName: "config") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.872478 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.872451 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.872620 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.872599 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.872839 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.872812 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5" (OuterVolumeSpecName: "kube-api-access-gc2m5") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "kube-api-access-gc2m5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:51:39.883231 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.883207 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config" (OuterVolumeSpecName: "web-config") pod "c6ea3bd1-90cc-4a17-890f-b61649e20482" (UID: "c6ea3bd1-90cc-4a17-890f-b61649e20482"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:39.969419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969380 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-db\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969410 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969421 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969430 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969439 2556 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-metrics-client-certs\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969449 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc2m5\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-kube-api-access-gc2m5\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969457 2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c6ea3bd1-90cc-4a17-890f-b61649e20482-config-out\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969466 2556 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-config\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969474 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969482 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969490 2556 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969498 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969506 2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969516 2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c6ea3bd1-90cc-4a17-890f-b61649e20482-tls-assets\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969523 2556 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-kube-rbac-proxy\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969532 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969539 2556 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-secret-grpc-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969549 2556 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-configmap-metrics-client-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969558 2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c6ea3bd1-90cc-4a17-890f-b61649e20482-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:39.969677 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:39.969566 2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c6ea3bd1-90cc-4a17-890f-b61649e20482-web-config\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 06 23:51:40.481045 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481017 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" exitCode=0 Jul 06 23:51:40.481045 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481039 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" exitCode=0 Jul 06 23:51:40.481045 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481046 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" exitCode=0 Jul 06 23:51:40.481045 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481052 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" exitCode=0 Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481058 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" exitCode=0 Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481063 2556 generic.go:356] "Generic (PLEG): container finished" podID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" exitCode=0 Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481093 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481111 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481209 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481236 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481246 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481260 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481287 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} Jul 06 23:51:40.481310 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.481311 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c6ea3bd1-90cc-4a17-890f-b61649e20482","Type":"ContainerDied","Data":"e0d3dd3b76835c9e4a5b383e70cc0a3eb24a365e231f893d35d7896b3de509ca"} Jul 06 23:51:40.489460 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.489446 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.496403 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.496380 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.502881 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.502864 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.505498 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.505479 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:40.510062 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.510039 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.510448 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.510425 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:40.516592 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.516575 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.523787 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.523594 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.530074 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530058 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.530342 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.530324 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.530405 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530353 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.530405 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530397 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.530638 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.530620 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.530688 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530647 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.530688 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530666 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.530907 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.530887 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.530977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530914 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.530977 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.530934 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.531190 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.531173 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.531230 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531195 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.531230 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531210 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.531453 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.531437 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.531511 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531455 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.531511 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531467 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.531676 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.531660 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.531713 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531683 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.531713 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.531697 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.532122 ip-10-0-132-169 kubenswrapper[2556]: E0706 23:51:40.532106 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.532159 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532126 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.532159 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532137 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.532343 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532324 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.532343 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532344 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.532593 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532575 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.532593 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532593 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.532834 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532813 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.532834 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.532833 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.533207 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533084 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.533207 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533109 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.533403 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533384 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.533466 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533405 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.533636 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533619 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.533685 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533636 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.533893 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533875 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.533953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.533892 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.534133 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534115 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.534182 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534135 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.534365 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534344 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.534426 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534365 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.534565 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534545 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.534624 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534567 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.534948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534728 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.534948 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534763 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.535065 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534983 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.535065 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.534997 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.535255 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535177 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.535255 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535198 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.535493 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535434 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.535493 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535449 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.535723 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535618 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.535723 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535651 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.535920 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535848 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.535920 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.535864 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.536127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536052 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.536127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536065 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.536423 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536294 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.536423 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536309 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.536625 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536491 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.536625 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536522 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.536791 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536771 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.536843 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.536791 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.537106 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537080 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.537186 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537107 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.537435 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537406 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.537435 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537428 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.537720 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537686 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.537720 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537720 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.537994 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537957 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.538062 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.537998 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.538255 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538234 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.538330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538257 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.538538 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538517 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.538601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538541 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.538876 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538858 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.538945 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.538877 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.539167 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539147 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.539167 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539167 2556 scope.go:117] "RemoveContainer" containerID="dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2" Jul 06 23:51:40.539421 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539396 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2"} err="failed to get container status \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": rpc error: code = NotFound desc = could not find container \"dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2\": container with ID starting with dfd4cd8daa80bb6bf2fd6e459e7c7b60fb6674ad2efe7df977073064e48330b2 not found: ID does not exist" Jul 06 23:51:40.539421 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539421 2556 scope.go:117] "RemoveContainer" containerID="342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334" Jul 06 23:51:40.539650 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539632 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334"} err="failed to get container status \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": rpc error: code = NotFound desc = could not find container \"342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334\": container with ID starting with 342cec4530b23df37fd5adaa4f6dc0a90993fb25a49acfcbc9aed1d131530334 not found: ID does not exist" Jul 06 23:51:40.539731 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539652 2556 scope.go:117] "RemoveContainer" containerID="4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec" Jul 06 23:51:40.539987 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539964 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec"} err="failed to get container status \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": rpc error: code = NotFound desc = could not find container \"4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec\": container with ID starting with 4e28fabb1bfd498a48181851252aebcb02b20fe7c38e1ff295fd349dabdcc2ec not found: ID does not exist" Jul 06 23:51:40.540074 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.539987 2556 scope.go:117] "RemoveContainer" containerID="6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47" Jul 06 23:51:40.540242 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540225 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47"} err="failed to get container status \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": rpc error: code = NotFound desc = could not find container \"6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47\": container with ID starting with 6452c7d04f3d6189fa29906446b0d2842fa1aa2eceba2f456cd280db6d295b47 not found: ID does not exist" Jul 06 23:51:40.540242 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540241 2556 scope.go:117] "RemoveContainer" containerID="eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f" Jul 06 23:51:40.540504 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540486 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f"} err="failed to get container status \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": rpc error: code = NotFound desc = could not find container \"eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f\": container with ID starting with eebc66798fca3d4bcdc3b585159b0a95ee97c3b8495415200f512ba0d0c2182f not found: ID does not exist" Jul 06 23:51:40.540504 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540503 2556 scope.go:117] "RemoveContainer" containerID="e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b" Jul 06 23:51:40.540759 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540724 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b"} err="failed to get container status \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": rpc error: code = NotFound desc = could not find container \"e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b\": container with ID starting with e415c589b29dd30440f318a3c19cdf3e4ebf045466a237d0690ab26e5f27230b not found: ID does not exist" Jul 06 23:51:40.540825 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.540795 2556 scope.go:117] "RemoveContainer" containerID="050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662" Jul 06 23:51:40.541060 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.541042 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662"} err="failed to get container status \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": rpc error: code = NotFound desc = could not find container \"050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662\": container with ID starting with 050cdfb723f7541d0c56935a086f56f0657bf7ffe6116e0789e0ac9ad3ccc662 not found: ID does not exist" Jul 06 23:51:40.543575 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543555 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:40.543869 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543855 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="thanos-sidecar" Jul 06 23:51:40.543869 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543869 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="thanos-sidecar" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543876 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543884 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543896 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="init-config-reloader" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543902 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="init-config-reloader" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543907 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-thanos" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543912 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-thanos" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543925 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="prometheus" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543931 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="prometheus" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543937 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-web" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543942 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-web" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543949 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="config-reloader" Jul 06 23:51:40.543953 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.543955 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="config-reloader" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544000 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-thanos" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544013 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="config-reloader" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544021 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544030 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="thanos-sidecar" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544036 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="kube-rbac-proxy-web" Jul 06 23:51:40.544330 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.544042 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" containerName="prometheus" Jul 06 23:51:40.549276 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.549261 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.552027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.552002 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 23:51:40.552295 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.552277 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-98wl7\"" Jul 06 23:51:40.552584 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.552565 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 23:51:40.552584 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.552578 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rv251l0o3mdl\"" Jul 06 23:51:40.555162 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555142 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 23:51:40.555162 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555155 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 23:51:40.555429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555186 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 23:51:40.555429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555240 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 23:51:40.555429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555387 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555452 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555539 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555560 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555607 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555741 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 23:51:40.556027 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.555780 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 23:51:40.558249 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.558231 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 23:51:40.559733 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.559715 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 23:51:40.565299 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.565277 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:40.676052 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676022 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676052 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676054 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676082 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676100 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-web-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676116 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cbkb\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-kube-api-access-8cbkb\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676185 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676220 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676284 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676265 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676297 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-config-out\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676329 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676415 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676439 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676462 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676491 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676567 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676538 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676923 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676923 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676611 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.676923 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.676646 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.777881 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777850 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777883 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-web-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777902 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cbkb\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-kube-api-access-8cbkb\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777939 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777956 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778054 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.777987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-config-out\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778169 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778218 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778278 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778306 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778330 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778371 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778358 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778384 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778408 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778444 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778475 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778518 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.778674 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.778578 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.779084 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.779059 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.779408 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.779183 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.779704 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.779674 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.780119 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.780097 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.781092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.780860 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8594e540-5e09-43d5-8276-a810b5aa9e8b-config-out\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.781185 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.781235 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.781372 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.781472 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.781702 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.782063 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-web-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.782233 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.782548 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.782957 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.782925 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.783456 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.782982 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.783687 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.783662 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.783884 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.783865 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-config\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.784091 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.784075 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8594e540-5e09-43d5-8276-a810b5aa9e8b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.784825 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.784806 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8594e540-5e09-43d5-8276-a810b5aa9e8b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.786091 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.786068 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cbkb\" (UniqueName: \"kubernetes.io/projected/8594e540-5e09-43d5-8276-a810b5aa9e8b-kube-api-access-8cbkb\") pod \"prometheus-k8s-0\" (UID: \"8594e540-5e09-43d5-8276-a810b5aa9e8b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.862608 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.862584 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:51:40.999646 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:40.999488 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:51:41.002265 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:51:41.002235 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8594e540_5e09_43d5_8276_a810b5aa9e8b.slice/crio-3a4b0d5a8ba33806e4ccad334f16936e7619c5e3e95d3fbb71082855e8d8b673 WatchSource:0}: Error finding container 3a4b0d5a8ba33806e4ccad334f16936e7619c5e3e95d3fbb71082855e8d8b673: Status 404 returned error can't find the container with id 3a4b0d5a8ba33806e4ccad334f16936e7619c5e3e95d3fbb71082855e8d8b673 Jul 06 23:51:41.487259 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:41.487164 2556 generic.go:356] "Generic (PLEG): container finished" podID="8594e540-5e09-43d5-8276-a810b5aa9e8b" containerID="31e8becc23e6cb4af94685fdba430362fb36b59a645a8d964229c29b82238a1f" exitCode=0 Jul 06 23:51:41.487259 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:41.487202 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerDied","Data":"31e8becc23e6cb4af94685fdba430362fb36b59a645a8d964229c29b82238a1f"} Jul 06 23:51:41.487259 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:41.487251 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"3a4b0d5a8ba33806e4ccad334f16936e7619c5e3e95d3fbb71082855e8d8b673"} Jul 06 23:51:41.590802 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:41.590772 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6ea3bd1-90cc-4a17-890f-b61649e20482" path="/var/lib/kubelet/pods/c6ea3bd1-90cc-4a17-890f-b61649e20482/volumes" Jul 06 23:51:42.497552 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497513 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"f460beaa7db3fdbe504d74c7a5a78e3e4e7cbd322f65c658a69aaaf058b06445"} Jul 06 23:51:42.497731 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"de173ab5af56d5058b4bbbff7b7750b4ad3040e65c93262b9082126baf816afc"} Jul 06 23:51:42.497731 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497682 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"1959e600eba40dd6f3cca4620e1fe20067553bf9f576bfcbefeafe48481bc0b5"} Jul 06 23:51:42.497731 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497716 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"dd73645daaf966757bc098d1fd489638efa8e942c9debdb9bc0a181052f0d9c0"} Jul 06 23:51:42.497852 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497770 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"811a433c37ad1988820859679ab6ae8ba4ed896013c3eaf5a3b6b65492461709"} Jul 06 23:51:42.497852 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.497814 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8594e540-5e09-43d5-8276-a810b5aa9e8b","Type":"ContainerStarted","Data":"6cb07f0016ad11786d5e738d22f9bd7e37c244ae020201de6f280ff4f612cdd9"} Jul 06 23:51:42.531720 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:42.531669 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.514721367 podStartE2EDuration="2.5316513s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.929933ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:51:42.528827765 +0000 UTC m=+169.458885268" watchObservedRunningTime="2026-07-06 23:51:42.5316513 +0000 UTC m=+169.461708805" Jul 06 23:51:45.863448 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:51:45.863364 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:52:40.863718 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:52:40.863686 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:52:40.879059 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:52:40.879037 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:52:41.736636 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:52:41.736606 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:53:53.467166 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:53:53.467138 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:53:53.467678 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:53:53.467197 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:53:53.470662 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:53:53.470642 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:53:53.470882 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:53:53.470802 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:53:53.478473 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:53:53.478455 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 23:55:20.713217 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.713183 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr"] Jul 06 23:55:20.716587 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.716572 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.719499 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.719479 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:55:20.720106 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.720090 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:55:20.720159 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.720091 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:55:20.720353 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.720338 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 23:55:20.720429 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.720380 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cs9gm\"" Jul 06 23:55:20.737190 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.737170 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr"] Jul 06 23:55:20.749028 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.749004 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.749127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.749047 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.749127 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.749088 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.749219 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.749123 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.749219 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.749148 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wpqp\" (UniqueName: \"kubernetes.io/projected/b095a823-55df-482e-81a6-6ba7315876da-kube-api-access-8wpqp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850143 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850112 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850268 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850147 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wpqp\" (UniqueName: \"kubernetes.io/projected/b095a823-55df-482e-81a6-6ba7315876da-kube-api-access-8wpqp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850268 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850212 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850268 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850262 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850431 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.850546 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.850523 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.852560 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.852531 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b095a823-55df-482e-81a6-6ba7315876da-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.852660 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.852583 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.852704 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.852673 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b095a823-55df-482e-81a6-6ba7315876da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:20.861640 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:20.861617 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wpqp\" (UniqueName: \"kubernetes.io/projected/b095a823-55df-482e-81a6-6ba7315876da-kube-api-access-8wpqp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-55bbr\" (UID: \"b095a823-55df-482e-81a6-6ba7315876da\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:21.025991 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:21.025968 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:21.204473 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:21.204401 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr"] Jul 06 23:55:21.205743 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:55:21.205720 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb095a823_55df_482e_81a6_6ba7315876da.slice/crio-ff554af926e3b4742835d2097d8ecf5a62d98cc5969a0627ab3351ad69672ba0 WatchSource:0}: Error finding container ff554af926e3b4742835d2097d8ecf5a62d98cc5969a0627ab3351ad69672ba0: Status 404 returned error can't find the container with id ff554af926e3b4742835d2097d8ecf5a62d98cc5969a0627ab3351ad69672ba0 Jul 06 23:55:21.207386 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:21.207367 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:55:21.326928 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:21.326848 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" event={"ID":"b095a823-55df-482e-81a6-6ba7315876da","Type":"ContainerStarted","Data":"ff554af926e3b4742835d2097d8ecf5a62d98cc5969a0627ab3351ad69672ba0"} Jul 06 23:55:24.342827 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:24.342742 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" event={"ID":"b095a823-55df-482e-81a6-6ba7315876da","Type":"ContainerStarted","Data":"95d5d693eb1a958579595c0a09d246b6af1b2142edcf9a0f2ef43efd95a0d3cf"} Jul 06 23:55:25.348435 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:25.348403 2556 generic.go:356] "Generic (PLEG): container finished" podID="b095a823-55df-482e-81a6-6ba7315876da" containerID="95d5d693eb1a958579595c0a09d246b6af1b2142edcf9a0f2ef43efd95a0d3cf" exitCode=0 Jul 06 23:55:25.348775 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:25.348470 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" event={"ID":"b095a823-55df-482e-81a6-6ba7315876da","Type":"ContainerDied","Data":"95d5d693eb1a958579595c0a09d246b6af1b2142edcf9a0f2ef43efd95a0d3cf"} Jul 06 23:55:26.355816 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:26.355776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" event={"ID":"b095a823-55df-482e-81a6-6ba7315876da","Type":"ContainerStarted","Data":"a27dd428e30668ccc0c8e04b72fac6cb5d4204d85289d113fb26ec71dae84ad4"} Jul 06 23:55:26.356275 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:26.355881 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:26.378021 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:26.377979 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" podStartSLOduration=2.086510815 podStartE2EDuration="6.377965663s" totalImagesPullingTime="3.206069612s" totalInitContainerRuntime="1.085385236s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:26.376062564 +0000 UTC m=+393.306120069" watchObservedRunningTime="2026-07-06 23:55:26.377965663 +0000 UTC m=+393.308023167" Jul 06 23:55:37.363186 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:37.363152 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-55bbr" Jul 06 23:55:46.397167 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.397127 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf"] Jul 06 23:55:46.400412 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.400396 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.403510 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.403486 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 23:55:46.404866 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.404845 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:55:46.404866 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.404855 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 23:55:46.405050 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.404922 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jknzk\"" Jul 06 23:55:46.405050 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.404958 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 23:55:46.405050 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.404958 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 23:55:46.409722 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.409703 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf"] Jul 06 23:55:46.458446 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.458411 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6673faf-b9c5-4e88-a28f-6717b3dbd060-manager-config\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.458601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.458460 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-metrics-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.458601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.458547 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp95d\" (UniqueName: \"kubernetes.io/projected/f6673faf-b9c5-4e88-a28f-6717b3dbd060-kube-api-access-wp95d\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.458601 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.458579 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.559765 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.559719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-metrics-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.559918 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.559794 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp95d\" (UniqueName: \"kubernetes.io/projected/f6673faf-b9c5-4e88-a28f-6717b3dbd060-kube-api-access-wp95d\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.559918 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.559828 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.559918 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.559855 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6673faf-b9c5-4e88-a28f-6717b3dbd060-manager-config\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.560479 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.560449 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6673faf-b9c5-4e88-a28f-6717b3dbd060-manager-config\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.562271 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.562248 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-metrics-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.562375 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.562289 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6673faf-b9c5-4e88-a28f-6717b3dbd060-cert\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.570074 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.570052 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp95d\" (UniqueName: \"kubernetes.io/projected/f6673faf-b9c5-4e88-a28f-6717b3dbd060-kube-api-access-wp95d\") pod \"lws-controller-manager-9d77cc8c4-z4lbf\" (UID: \"f6673faf-b9c5-4e88-a28f-6717b3dbd060\") " pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.710730 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.710638 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:46.844779 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:46.844740 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf"] Jul 06 23:55:46.847140 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:55:46.847111 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6673faf_b9c5_4e88_a28f_6717b3dbd060.slice/crio-c17daa21850ef81e07e9d9bbad3c7060f09df7bbd723094b4c7b9604badd36a2 WatchSource:0}: Error finding container c17daa21850ef81e07e9d9bbad3c7060f09df7bbd723094b4c7b9604badd36a2: Status 404 returned error can't find the container with id c17daa21850ef81e07e9d9bbad3c7060f09df7bbd723094b4c7b9604badd36a2 Jul 06 23:55:47.452279 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:47.452245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" event={"ID":"f6673faf-b9c5-4e88-a28f-6717b3dbd060","Type":"ContainerStarted","Data":"c17daa21850ef81e07e9d9bbad3c7060f09df7bbd723094b4c7b9604badd36a2"} Jul 06 23:55:49.463883 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:49.463802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" event={"ID":"f6673faf-b9c5-4e88-a28f-6717b3dbd060","Type":"ContainerStarted","Data":"882f73909abc0586a1e9c59e4ded5892719aa296e2097df29fa47add18cf9361"} Jul 06 23:55:49.464244 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:49.463936 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:55:49.483758 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:49.483706 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" podStartSLOduration=1.119490761 podStartE2EDuration="3.483690092s" totalImagesPullingTime="2.364199331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:49.483011613 +0000 UTC m=+416.413069118" watchObservedRunningTime="2026-07-06 23:55:49.483690092 +0000 UTC m=+416.413747595" Jul 06 23:55:50.073472 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.073441 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj"] Jul 06 23:55:50.076697 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.076681 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.079656 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.079634 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-48vmd\"" Jul 06 23:55:50.079810 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.079724 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 23:55:50.079810 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.079741 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 23:55:50.079810 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.079791 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 23:55:50.079810 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.079759 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 23:55:50.090037 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.090011 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj"] Jul 06 23:55:50.191000 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.190977 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/462f541c-54a8-41d2-abf1-fbcb1df4e80c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191010 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191030 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr87t\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-kube-api-access-hr87t\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191109 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191298 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191119 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191298 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191183 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.191298 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.191229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.291994 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.291969 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/462f541c-54a8-41d2-abf1-fbcb1df4e80c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292003 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292026 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr87t\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-kube-api-access-hr87t\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292044 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292063 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292100 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292087 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292314 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.292841 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.292811 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.294322 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.294296 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/462f541c-54a8-41d2-abf1-fbcb1df4e80c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.294574 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.294552 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.294614 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.294560 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.294614 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.294562 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.317176 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.317147 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.321334 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.321309 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr87t\" (UniqueName: \"kubernetes.io/projected/462f541c-54a8-41d2-abf1-fbcb1df4e80c-kube-api-access-hr87t\") pod \"istiod-openshift-gateway-94bb8fbfd-99ckj\" (UID: \"462f541c-54a8-41d2-abf1-fbcb1df4e80c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.385403 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.385357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:50.547401 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:50.547369 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj"] Jul 06 23:55:50.553683 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:55:50.553658 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462f541c_54a8_41d2_abf1_fbcb1df4e80c.slice/crio-78c90061cd05f2290b09c43a322704cf129c7f61608c6746420642345bf4c148 WatchSource:0}: Error finding container 78c90061cd05f2290b09c43a322704cf129c7f61608c6746420642345bf4c148: Status 404 returned error can't find the container with id 78c90061cd05f2290b09c43a322704cf129c7f61608c6746420642345bf4c148 Jul 06 23:55:51.476905 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:51.476859 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" event={"ID":"462f541c-54a8-41d2-abf1-fbcb1df4e80c","Type":"ContainerStarted","Data":"78c90061cd05f2290b09c43a322704cf129c7f61608c6746420642345bf4c148"} Jul 06 23:55:52.805016 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:52.804974 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:52.805303 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:52.805042 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:53.487944 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:53.487905 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" event={"ID":"462f541c-54a8-41d2-abf1-fbcb1df4e80c","Type":"ContainerStarted","Data":"cf0226bfa60de94b890609a2635845db1f1ab367c2598b61410a74198d8d991c"} Jul 06 23:55:53.488125 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:53.488084 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:55:53.489685 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:53.489653 2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-99ckj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 23:55:53.489835 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:53.489720 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" podUID="462f541c-54a8-41d2-abf1-fbcb1df4e80c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 23:55:53.566537 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:53.566479 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" podStartSLOduration=1.31733036 podStartE2EDuration="3.566455763s" totalImagesPullingTime="2.249125403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:53.563543969 +0000 UTC m=+420.493601489" watchObservedRunningTime="2026-07-06 23:55:53.566455763 +0000 UTC m=+420.496513280" Jul 06 23:55:54.492479 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:55:54.492450 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-99ckj" Jul 06 23:56:00.472332 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:56:00.472301 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9d77cc8c4-z4lbf" Jul 06 23:58:07.466193 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.466113 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:58:07.468483 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.468466 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.474092 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.474070 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fhz99\"" Jul 06 23:58:07.474205 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.474136 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:58:07.475357 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.475339 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:58:07.475456 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.475388 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 23:58:07.480907 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.480884 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:58:07.498615 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.498583 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:58:07.648064 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.648032 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4xwp\" (UniqueName: \"kubernetes.io/projected/66c18bcd-10ed-4637-b833-28d72a81a9af-kube-api-access-x4xwp\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.648227 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.648148 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66c18bcd-10ed-4637-b833-28d72a81a9af-config-file\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.749281 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.749250 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4xwp\" (UniqueName: \"kubernetes.io/projected/66c18bcd-10ed-4637-b833-28d72a81a9af-kube-api-access-x4xwp\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.749448 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.749352 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66c18bcd-10ed-4637-b833-28d72a81a9af-config-file\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.750048 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.750025 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66c18bcd-10ed-4637-b833-28d72a81a9af-config-file\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.757162 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.757133 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4xwp\" (UniqueName: \"kubernetes.io/projected/66c18bcd-10ed-4637-b833-28d72a81a9af-kube-api-access-x4xwp\") pod \"limitador-limitador-54d6d988d5-8bflg\" (UID: \"66c18bcd-10ed-4637-b833-28d72a81a9af\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.778511 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.778494 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:07.911346 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:07.911315 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:58:07.915326 ip-10-0-132-169 kubenswrapper[2556]: W0706 23:58:07.915299 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c18bcd_10ed_4637_b833_28d72a81a9af.slice/crio-da7ee24a1ad4a0cd3ecb9172d8b68e6fa9e5574f589b9be5ddc8020a4ac9f60b WatchSource:0}: Error finding container da7ee24a1ad4a0cd3ecb9172d8b68e6fa9e5574f589b9be5ddc8020a4ac9f60b: Status 404 returned error can't find the container with id da7ee24a1ad4a0cd3ecb9172d8b68e6fa9e5574f589b9be5ddc8020a4ac9f60b Jul 06 23:58:08.064039 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:08.063955 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" event={"ID":"66c18bcd-10ed-4637-b833-28d72a81a9af","Type":"ContainerStarted","Data":"da7ee24a1ad4a0cd3ecb9172d8b68e6fa9e5574f589b9be5ddc8020a4ac9f60b"} Jul 06 23:58:11.079785 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:11.079732 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" event={"ID":"66c18bcd-10ed-4637-b833-28d72a81a9af","Type":"ContainerStarted","Data":"2a7110db4f8e6476a4bdb3f6d6fccd17ffe3e79afe153803e93201c471874871"} Jul 06 23:58:11.080175 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:11.079880 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:11.099190 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:11.099142 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" podStartSLOduration=1.683690637 podStartE2EDuration="4.099128025s" totalImagesPullingTime="2.415437388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:11.097979592 +0000 UTC m=+558.028037096" watchObservedRunningTime="2026-07-06 23:58:11.099128025 +0000 UTC m=+558.029185530" Jul 06 23:58:22.086030 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:22.086000 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8bflg" Jul 06 23:58:53.495638 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:53.495610 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:58:53.496289 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:53.495965 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 06 23:58:53.498419 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:53.498402 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:58:53.498831 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:58:53.498812 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 06 23:59:22.244495 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:59:22.244410 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:59:52.169968 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:59:52.169933 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 06 23:59:58.169324 ip-10-0-132-169 kubenswrapper[2556]: I0706 23:59:58.169290 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:00:02.574169 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:02.574133 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:00:13.377640 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.377605 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h"] Jul 07 00:00:13.381392 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.381365 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.385288 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.385267 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 00:00:13.385409 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.385315 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pfpg\"" Jul 07 00:00:13.385409 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.385378 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 00:00:13.385409 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.385382 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 00:00:13.390068 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.390044 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h"] Jul 07 00:00:13.563842 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24k2p\" (UniqueName: \"kubernetes.io/projected/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-kube-api-access-24k2p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.564013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563841 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.564013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563875 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.564013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563906 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.564013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563927 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.564013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.563969 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665087 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665116 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665155 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665186 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665224 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24k2p\" (UniqueName: \"kubernetes.io/projected/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-kube-api-access-24k2p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665583 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665563 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665642 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665600 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.665675 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.665631 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.667271 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.667250 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.667586 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.667570 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.673498 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.673480 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24k2p\" (UniqueName: \"kubernetes.io/projected/13848a88-dc5c-46b8-b00c-eaa0d5d1c76c-kube-api-access-24k2p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h\" (UID: \"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.692340 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.692323 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:13.818088 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:13.817926 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h"] Jul 07 00:00:13.820727 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:00:13.820692 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13848a88_dc5c_46b8_b00c_eaa0d5d1c76c.slice/crio-0632350796b39005f9195b711e0d4868c6eed2dc35c682ece386e1659bccdfde WatchSource:0}: Error finding container 0632350796b39005f9195b711e0d4868c6eed2dc35c682ece386e1659bccdfde: Status 404 returned error can't find the container with id 0632350796b39005f9195b711e0d4868c6eed2dc35c682ece386e1659bccdfde Jul 07 00:00:14.622824 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:14.622793 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" event={"ID":"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c","Type":"ContainerStarted","Data":"0632350796b39005f9195b711e0d4868c6eed2dc35c682ece386e1659bccdfde"} Jul 07 00:00:15.077478 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:15.077444 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:00:27.179456 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:27.179421 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:00:38.725338 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:38.725299 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" event={"ID":"13848a88-dc5c-46b8-b00c-eaa0d5d1c76c","Type":"ContainerStarted","Data":"9c084461a4086df125f773b4ee751ce8f038a70ec9e97d2d20401baff2dc87c9"} Jul 07 00:00:38.743546 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:38.743501 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" podStartSLOduration=1.342341812 podStartE2EDuration="25.743486305s" totalImagesPullingTime="24.401144493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:38.741526417 +0000 UTC m=+705.671583922" watchObservedRunningTime="2026-07-07 00:00:38.743486305 +0000 UTC m=+705.673543786" Jul 07 00:00:43.692480 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:43.692440 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:43.693084 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:43.692509 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:43.707321 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:43.707295 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:43.765653 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:43.765623 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h" Jul 07 00:00:50.582555 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:00:50.582468 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:02:06.473605 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:02:06.473563 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:02:16.469462 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:02:16.469379 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:02:26.865082 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:02:26.865046 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:02:36.679910 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:02:36.679875 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:02:47.676320 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:02:47.676285 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:03:17.430400 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:17.430366 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:03:29.971922 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:29.971887 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:03:53.522965 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:53.522893 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:03:53.524086 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:53.524064 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:03:53.526402 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:53.526385 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:03:53.527338 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:03:53.527322 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:04:02.476142 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:02.476107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:04:11.076241 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:11.076206 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:04:20.976013 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:20.975978 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:04:28.870888 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:28.870857 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:04:38.867208 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:38.867174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:04:54.172853 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:04:54.172813 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:05:00.565971 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:05:00.565936 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:05:08.068508 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:05:08.068468 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:05:31.670224 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:05:31.670143 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:05:40.471925 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:05:40.471895 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:03.869149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:03.869111 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:15.969265 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:15.969226 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:31.076534 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:31.076499 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:40.976271 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:40.976235 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:50.170937 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:50.170854 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:06:59.375463 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:06:59.375428 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:07:06.179364 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:07:06.179328 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:08:53.550320 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:08:53.550293 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:08:53.553235 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:08:53.553213 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:08:53.553627 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:08:53.553609 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:08:53.556424 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:08:53.556408 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:08:56.469133 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:08:56.469098 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:09:05.471250 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:09:05.471207 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:09:17.770149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:09:17.770114 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:09:30.068694 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:09:30.068656 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:09:42.174514 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:09:42.174480 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:09:51.068986 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:09:51.068901 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:10:00.174969 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:10:00.174934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:10:11.071190 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:10:11.071156 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:10:22.379231 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:10:22.379187 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:10:31.068847 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:10:31.068799 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:10:42.065114 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:10:42.065083 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:06.275769 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:06.275716 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:13.379130 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:13.379098 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:22.989847 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:22.989768 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:33.577012 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:33.576979 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:41.271664 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:41.271623 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:11:54.174665 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:11:54.174626 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:03.277617 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:03.277584 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:12.184437 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:12.184398 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:22.383558 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:22.383521 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:31.176409 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:31.176380 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:40.482715 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:40.482683 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:12:53.782075 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:12:53.782000 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:02.778560 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:02.778518 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:11.371683 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:11.371637 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:20.278480 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:20.278438 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:26.272840 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:26.272805 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:38.271664 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:38.271626 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:47.577084 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:47.577049 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:13:53.580098 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:53.580065 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:13:53.583040 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:53.583016 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:13:53.585224 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:53.585198 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:13:53.588538 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:53.588519 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:13:59.767066 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:13:59.767031 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:11.772201 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:11.772165 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:24.779250 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:24.779162 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:36.879534 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:36.879499 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:44.873876 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:44.873840 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:55.371333 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:55.371226 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:14:57.576238 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.576204 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:14:57.579508 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.579489 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.582141 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.582118 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-aab879b8-serving-cert\"" Jul 07 00:14:57.582248 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.582122 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:14:57.592201 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.592179 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:14:57.624113 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.624085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.624219 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.624126 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.624265 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.624237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dcnr\" (UniqueName: \"kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.725454 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.725416 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.725598 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.725488 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dcnr\" (UniqueName: \"kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.725598 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.725537 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.725598 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:14:57.725552 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aab879b8-serving-cert: secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:14:57.725698 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:14:57.725627 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls podName:45178236-3b77-4cfb-9e5d-a5b7e3efb96d nodeName:}" failed. No retries permitted until 2026-07-07 00:14:58.225608855 +0000 UTC m=+1565.155666337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls") pod "maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d") : secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:14:57.726174 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.726158 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:57.734401 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:57.734381 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dcnr\" (UniqueName: \"kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:58.229665 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:58.229628 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:58.229855 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:14:58.229801 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aab879b8-serving-cert: secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:14:58.229900 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:14:58.229881 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls podName:45178236-3b77-4cfb-9e5d-a5b7e3efb96d nodeName:}" failed. No retries permitted until 2026-07-07 00:14:59.229865271 +0000 UTC m=+1566.159922753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls") pod "maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d") : secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:14:59.238902 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:59.238871 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:59.241160 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:59.241138 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:59.391393 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:59.391360 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:14:59.533937 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:59.533904 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:14:59.536740 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:14:59.536715 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45178236_3b77_4cfb_9e5d_a5b7e3efb96d.slice/crio-9deecd87b3859b7ee9970d62da9fe689fa198231ee1ca8b24adca3ec37f1766a WatchSource:0}: Error finding container 9deecd87b3859b7ee9970d62da9fe689fa198231ee1ca8b24adca3ec37f1766a: Status 404 returned error can't find the container with id 9deecd87b3859b7ee9970d62da9fe689fa198231ee1ca8b24adca3ec37f1766a Jul 07 00:14:59.538135 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:14:59.538115 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:15:00.173468 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.173436 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-rbf8n"] Jul 07 00:15:00.177808 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.177781 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:00.206004 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.205970 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-rbf8n"] Jul 07 00:15:00.247368 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.247333 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcbrr\" (UniqueName: \"kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr\") pod \"maas-api-key-cleanup-29723055-rbf8n\" (UID: \"30826312-2250-4686-9100-868b4b1ff9c3\") " pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:00.348131 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.348044 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcbrr\" (UniqueName: \"kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr\") pod \"maas-api-key-cleanup-29723055-rbf8n\" (UID: \"30826312-2250-4686-9100-868b4b1ff9c3\") " pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:00.357864 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.357822 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcbrr\" (UniqueName: \"kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr\") pod \"maas-api-key-cleanup-29723055-rbf8n\" (UID: \"30826312-2250-4686-9100-868b4b1ff9c3\") " pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:00.488763 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.488711 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:00.533917 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.533843 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" event={"ID":"45178236-3b77-4cfb-9e5d-a5b7e3efb96d","Type":"ContainerStarted","Data":"9deecd87b3859b7ee9970d62da9fe689fa198231ee1ca8b24adca3ec37f1766a"} Jul 07 00:15:00.642996 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:00.642859 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-rbf8n"] Jul 07 00:15:00.645772 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:15:00.645727 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30826312_2250_4686_9100_868b4b1ff9c3.slice/crio-db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55 WatchSource:0}: Error finding container db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55: Status 404 returned error can't find the container with id db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55 Jul 07 00:15:01.541021 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:01.540982 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" event={"ID":"30826312-2250-4686-9100-868b4b1ff9c3","Type":"ContainerStarted","Data":"db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55"} Jul 07 00:15:02.546875 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:02.546828 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" event={"ID":"45178236-3b77-4cfb-9e5d-a5b7e3efb96d","Type":"ContainerStarted","Data":"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828"} Jul 07 00:15:02.547347 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:02.546899 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:15:02.569603 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:02.569533 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" podStartSLOduration=3.140742064 podStartE2EDuration="5.569514947s" totalImagesPullingTime="2.428772883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:02.566761225 +0000 UTC m=+1569.496818720" watchObservedRunningTime="2026-07-07 00:15:02.569514947 +0000 UTC m=+1569.499572451" Jul 07 00:15:03.099511 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:03.099478 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:15:03.579523 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:03.579481 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aab879b8-serving-cert: secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:15:03.579974 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:03.579551 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls podName:45178236-3b77-4cfb-9e5d-a5b7e3efb96d nodeName:}" failed. No retries permitted until 2026-07-07 00:15:04.079535247 +0000 UTC m=+1571.009592729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls") pod "maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d") : secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:15:04.084691 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:04.084660 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aab879b8-serving-cert: secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:15:04.084885 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:04.084729 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls podName:45178236-3b77-4cfb-9e5d-a5b7e3efb96d nodeName:}" failed. No retries permitted until 2026-07-07 00:15:05.08471421 +0000 UTC m=+1572.014771692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls") pod "maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d") : secret "maas-api-e2e-ait-aab879b8-serving-cert" not found Jul 07 00:15:04.558961 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.558926 2556 generic.go:356] "Generic (PLEG): container finished" podID="30826312-2250-4686-9100-868b4b1ff9c3" containerID="49349829ad8086fe1818a38502afd9c1126195a0fdec425f30fa171cba62600a" exitCode=0 Jul 07 00:15:04.559127 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.558997 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" event={"ID":"30826312-2250-4686-9100-868b4b1ff9c3","Type":"ContainerDied","Data":"49349829ad8086fe1818a38502afd9c1126195a0fdec425f30fa171cba62600a"} Jul 07 00:15:04.559127 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.559105 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" podUID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" containerName="maas-api" containerID="cri-o://9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828" gracePeriod=30 Jul 07 00:15:04.842350 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.842321 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:15:04.892420 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.892392 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dcnr\" (UniqueName: \"kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr\") pod \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " Jul 07 00:15:04.892540 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.892512 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca\") pod \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " Jul 07 00:15:04.892588 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.892565 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") pod \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\" (UID: \"45178236-3b77-4cfb-9e5d-a5b7e3efb96d\") " Jul 07 00:15:04.892847 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.892824 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45178236-3b77-4cfb-9e5d-a5b7e3efb96d" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:04.894537 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.894518 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr" (OuterVolumeSpecName: "kube-api-access-7dcnr") pod "45178236-3b77-4cfb-9e5d-a5b7e3efb96d" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d"). InnerVolumeSpecName "kube-api-access-7dcnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:04.894645 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.894599 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45178236-3b77-4cfb-9e5d-a5b7e3efb96d" (UID: "45178236-3b77-4cfb-9e5d-a5b7e3efb96d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:15:04.999275 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.994177 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:04.999275 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.994211 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:04.999275 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:04.994226 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dcnr\" (UniqueName: \"kubernetes.io/projected/45178236-3b77-4cfb-9e5d-a5b7e3efb96d-kube-api-access-7dcnr\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:05.565824 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.565789 2556 generic.go:356] "Generic (PLEG): container finished" podID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" containerID="9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828" exitCode=0 Jul 07 00:15:05.565824 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.565832 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" Jul 07 00:15:05.566061 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.565880 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" event={"ID":"45178236-3b77-4cfb-9e5d-a5b7e3efb96d","Type":"ContainerDied","Data":"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828"} Jul 07 00:15:05.566061 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.565953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn" event={"ID":"45178236-3b77-4cfb-9e5d-a5b7e3efb96d","Type":"ContainerDied","Data":"9deecd87b3859b7ee9970d62da9fe689fa198231ee1ca8b24adca3ec37f1766a"} Jul 07 00:15:05.566061 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.565976 2556 scope.go:117] "RemoveContainer" containerID="9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828" Jul 07 00:15:05.575996 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.575980 2556 scope.go:117] "RemoveContainer" containerID="9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828" Jul 07 00:15:05.576277 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:05.576259 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828\": container with ID starting with 9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828 not found: ID does not exist" containerID="9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828" Jul 07 00:15:05.576355 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.576288 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828"} err="failed to get container status \"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828\": rpc error: code = NotFound desc = could not find container \"9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828\": container with ID starting with 9cc54eed6e82927a7cabbde4cf3c89d9554ddd6b4f0ac543907e97fc20ef5828 not found: ID does not exist" Jul 07 00:15:05.590413 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.590369 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:15:05.591951 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.591933 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-aab879b8-68954d6c5f-5zrxn"] Jul 07 00:15:05.698557 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.698537 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:05.800579 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.800546 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcbrr\" (UniqueName: \"kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr\") pod \"30826312-2250-4686-9100-868b4b1ff9c3\" (UID: \"30826312-2250-4686-9100-868b4b1ff9c3\") " Jul 07 00:15:05.802651 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.802618 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr" (OuterVolumeSpecName: "kube-api-access-bcbrr") pod "30826312-2250-4686-9100-868b4b1ff9c3" (UID: "30826312-2250-4686-9100-868b4b1ff9c3"). InnerVolumeSpecName "kube-api-access-bcbrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:05.901690 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:05.901616 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcbrr\" (UniqueName: \"kubernetes.io/projected/30826312-2250-4686-9100-868b4b1ff9c3-kube-api-access-bcbrr\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:06.572219 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:06.572137 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" Jul 07 00:15:06.572219 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:06.572167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-rbf8n" event={"ID":"30826312-2250-4686-9100-868b4b1ff9c3","Type":"ContainerDied","Data":"db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55"} Jul 07 00:15:06.572422 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:06.572220 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db01dd961c70031818e2e0e3d919587db17541a9052e8c7aea20fdc7d93c8b55" Jul 07 00:15:07.588595 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:07.588556 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" path="/var/lib/kubelet/pods/45178236-3b77-4cfb-9e5d-a5b7e3efb96d/volumes" Jul 07 00:15:10.160870 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.160839 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:10.161241 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161175 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30826312-2250-4686-9100-868b4b1ff9c3" containerName="cleanup" Jul 07 00:15:10.161241 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161187 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="30826312-2250-4686-9100-868b4b1ff9c3" containerName="cleanup" Jul 07 00:15:10.161241 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161202 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" containerName="maas-api" Jul 07 00:15:10.161241 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161208 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" containerName="maas-api" Jul 07 00:15:10.161380 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161269 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="45178236-3b77-4cfb-9e5d-a5b7e3efb96d" containerName="maas-api" Jul 07 00:15:10.161380 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.161281 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="30826312-2250-4686-9100-868b4b1ff9c3" containerName="cleanup" Jul 07 00:15:10.164415 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.164395 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.167004 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.166966 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-4c7f79fd-serving-cert\"" Jul 07 00:15:10.167004 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.166978 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:15:10.175301 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.175278 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:10.239150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.239122 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc9cd\" (UniqueName: \"kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.239259 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.239159 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.239259 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.239219 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.340556 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.340520 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc9cd\" (UniqueName: \"kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.340663 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.340564 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.340663 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.340596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.340767 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:10.340704 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-4c7f79fd-serving-cert: secret "maas-api-e2e-ait-4c7f79fd-serving-cert" not found Jul 07 00:15:10.340816 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:10.340789 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls podName:03376c03-60bb-4ec1-9bfa-5fbe6aa06f99 nodeName:}" failed. No retries permitted until 2026-07-07 00:15:10.840769973 +0000 UTC m=+1577.770827472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls") pod "maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" (UID: "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99") : secret "maas-api-e2e-ait-4c7f79fd-serving-cert" not found Jul 07 00:15:10.341217 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.341197 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.351534 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.351514 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc9cd\" (UniqueName: \"kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.845386 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.845356 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:10.847784 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:10.847733 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") pod \"maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:11.075574 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:11.075544 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:11.206465 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:11.206436 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:11.209539 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:15:11.209506 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03376c03_60bb_4ec1_9bfa_5fbe6aa06f99.slice/crio-ac358e0675e0a1d7b35c62ab0eb0331a4b26d5a0697cdd470ddd70185225f858 WatchSource:0}: Error finding container ac358e0675e0a1d7b35c62ab0eb0331a4b26d5a0697cdd470ddd70185225f858: Status 404 returned error can't find the container with id ac358e0675e0a1d7b35c62ab0eb0331a4b26d5a0697cdd470ddd70185225f858 Jul 07 00:15:11.602647 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:11.602613 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" event={"ID":"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99","Type":"ContainerStarted","Data":"ac358e0675e0a1d7b35c62ab0eb0331a4b26d5a0697cdd470ddd70185225f858"} Jul 07 00:15:12.607911 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:12.607874 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" event={"ID":"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99","Type":"ContainerStarted","Data":"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607"} Jul 07 00:15:12.608308 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:12.607938 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:12.625505 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:12.625456 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" podStartSLOduration=2.158735392 podStartE2EDuration="2.625440498s" totalImagesPullingTime="466.705106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:12.62423277 +0000 UTC m=+1579.554290274" watchObservedRunningTime="2026-07-07 00:15:12.625440498 +0000 UTC m=+1579.555498006" Jul 07 00:15:13.368578 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:13.368540 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:15:15.697448 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.697414 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:15.697864 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.697601 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" podUID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" containerName="maas-api" containerID="cri-o://9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607" gracePeriod=30 Jul 07 00:15:15.936148 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.936122 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:15.991379 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.991353 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc9cd\" (UniqueName: \"kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd\") pod \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " Jul 07 00:15:15.991509 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.991407 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") pod \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " Jul 07 00:15:15.991509 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.991472 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca\") pod \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\" (UID: \"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99\") " Jul 07 00:15:15.991868 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.991841 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" (UID: "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:15.993378 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.993350 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" (UID: "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:15:15.993499 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:15.993471 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd" (OuterVolumeSpecName: "kube-api-access-sc9cd") pod "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" (UID: "03376c03-60bb-4ec1-9bfa-5fbe6aa06f99"). InnerVolumeSpecName "kube-api-access-sc9cd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:16.092086 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.092053 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:16.092086 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.092080 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:16.092086 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.092090 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc9cd\" (UniqueName: \"kubernetes.io/projected/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99-kube-api-access-sc9cd\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:16.625580 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.625542 2556 generic.go:356] "Generic (PLEG): container finished" podID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" containerID="9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607" exitCode=0 Jul 07 00:15:16.625767 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.625599 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" Jul 07 00:15:16.625767 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.625604 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" event={"ID":"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99","Type":"ContainerDied","Data":"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607"} Jul 07 00:15:16.625767 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.625648 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr" event={"ID":"03376c03-60bb-4ec1-9bfa-5fbe6aa06f99","Type":"ContainerDied","Data":"ac358e0675e0a1d7b35c62ab0eb0331a4b26d5a0697cdd470ddd70185225f858"} Jul 07 00:15:16.625767 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.625668 2556 scope.go:117] "RemoveContainer" containerID="9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607" Jul 07 00:15:16.635926 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.635907 2556 scope.go:117] "RemoveContainer" containerID="9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607" Jul 07 00:15:16.636184 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:16.636166 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607\": container with ID starting with 9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607 not found: ID does not exist" containerID="9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607" Jul 07 00:15:16.636249 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.636195 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607"} err="failed to get container status \"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607\": rpc error: code = NotFound desc = could not find container \"9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607\": container with ID starting with 9688cd71922492f4970bdc95336a813fe78a07b9a1d21a8c102d5e6809487607 not found: ID does not exist" Jul 07 00:15:16.647668 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.647644 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:16.649764 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:16.649731 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-4c7f79fd-969d5bbc-px4fr"] Jul 07 00:15:17.590206 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:17.590171 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" path="/var/lib/kubelet/pods/03376c03-60bb-4ec1-9bfa-5fbe6aa06f99/volumes" Jul 07 00:15:24.772788 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:24.772736 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:15:29.088585 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.088543 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:29.089164 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.089144 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" containerName="maas-api" Jul 07 00:15:29.089246 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.089167 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" containerName="maas-api" Jul 07 00:15:29.089300 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.089252 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="03376c03-60bb-4ec1-9bfa-5fbe6aa06f99" containerName="maas-api" Jul 07 00:15:29.097158 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.097136 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.100076 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.100052 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2a6aa737-serving-cert\"" Jul 07 00:15:29.100188 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.100086 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:15:29.101352 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.101330 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:29.200717 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.200684 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.200890 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.200773 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.200890 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.200801 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf57d\" (UniqueName: \"kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.301979 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.301949 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.302120 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.301983 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf57d\" (UniqueName: \"kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.302120 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.302037 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.302120 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:29.302098 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-2a6aa737-serving-cert: secret "maas-api-e2e-derive-2a6aa737-serving-cert" not found Jul 07 00:15:29.302242 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:29.302174 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls podName:20095afa-72bf-47ef-9b46-5c93bbaef4ed nodeName:}" failed. No retries permitted until 2026-07-07 00:15:29.802156957 +0000 UTC m=+1596.732214442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls") pod "maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" (UID: "20095afa-72bf-47ef-9b46-5c93bbaef4ed") : secret "maas-api-e2e-derive-2a6aa737-serving-cert" not found Jul 07 00:15:29.302724 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.302708 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.314172 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.314144 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf57d\" (UniqueName: \"kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.806716 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.806684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:29.809079 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:29.809061 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") pod \"maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:30.008764 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.008721 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:30.139646 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.139601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:30.141879 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:15:30.141851 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20095afa_72bf_47ef_9b46_5c93bbaef4ed.slice/crio-3883b1f9a79273d66f088fba51e550caf554a5be55262eafd9fb0a5a0f8ffea1 WatchSource:0}: Error finding container 3883b1f9a79273d66f088fba51e550caf554a5be55262eafd9fb0a5a0f8ffea1: Status 404 returned error can't find the container with id 3883b1f9a79273d66f088fba51e550caf554a5be55262eafd9fb0a5a0f8ffea1 Jul 07 00:15:30.696449 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.696421 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" event={"ID":"20095afa-72bf-47ef-9b46-5c93bbaef4ed","Type":"ContainerStarted","Data":"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2"} Jul 07 00:15:30.696600 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.696470 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" event={"ID":"20095afa-72bf-47ef-9b46-5c93bbaef4ed","Type":"ContainerStarted","Data":"3883b1f9a79273d66f088fba51e550caf554a5be55262eafd9fb0a5a0f8ffea1"} Jul 07 00:15:30.696600 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.696497 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:30.714187 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:30.714144 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" podStartSLOduration=1.313399524 podStartE2EDuration="1.714130387s" totalImagesPullingTime="400.730863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:30.712429004 +0000 UTC m=+1597.642486508" watchObservedRunningTime="2026-07-07 00:15:30.714130387 +0000 UTC m=+1597.644187891" Jul 07 00:15:33.485404 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:33.485366 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:15:33.794871 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:33.794834 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:33.795076 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:33.795054 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" podUID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" containerName="maas-api" containerID="cri-o://30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2" gracePeriod=30 Jul 07 00:15:34.049742 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.049684 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:34.139029 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.138989 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") pod \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " Jul 07 00:15:34.139196 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.139058 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca\") pod \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " Jul 07 00:15:34.139196 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.139085 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf57d\" (UniqueName: \"kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d\") pod \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\" (UID: \"20095afa-72bf-47ef-9b46-5c93bbaef4ed\") " Jul 07 00:15:34.139436 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.139412 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20095afa-72bf-47ef-9b46-5c93bbaef4ed" (UID: "20095afa-72bf-47ef-9b46-5c93bbaef4ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:34.141178 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.141155 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d" (OuterVolumeSpecName: "kube-api-access-hf57d") pod "20095afa-72bf-47ef-9b46-5c93bbaef4ed" (UID: "20095afa-72bf-47ef-9b46-5c93bbaef4ed"). InnerVolumeSpecName "kube-api-access-hf57d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:34.141233 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.141159 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20095afa-72bf-47ef-9b46-5c93bbaef4ed" (UID: "20095afa-72bf-47ef-9b46-5c93bbaef4ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:15:34.240410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.240373 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20095afa-72bf-47ef-9b46-5c93bbaef4ed-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:34.240410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.240408 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20095afa-72bf-47ef-9b46-5c93bbaef4ed-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:34.240410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.240418 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf57d\" (UniqueName: \"kubernetes.io/projected/20095afa-72bf-47ef-9b46-5c93bbaef4ed-kube-api-access-hf57d\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:15:34.716419 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.716388 2556 generic.go:356] "Generic (PLEG): container finished" podID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" containerID="30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2" exitCode=0 Jul 07 00:15:34.716853 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.716448 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" Jul 07 00:15:34.716853 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.716480 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" event={"ID":"20095afa-72bf-47ef-9b46-5c93bbaef4ed","Type":"ContainerDied","Data":"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2"} Jul 07 00:15:34.716853 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.716533 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h" event={"ID":"20095afa-72bf-47ef-9b46-5c93bbaef4ed","Type":"ContainerDied","Data":"3883b1f9a79273d66f088fba51e550caf554a5be55262eafd9fb0a5a0f8ffea1"} Jul 07 00:15:34.716853 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.716554 2556 scope.go:117] "RemoveContainer" containerID="30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2" Jul 07 00:15:34.725420 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.725403 2556 scope.go:117] "RemoveContainer" containerID="30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2" Jul 07 00:15:34.725655 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:15:34.725633 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2\": container with ID starting with 30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2 not found: ID does not exist" containerID="30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2" Jul 07 00:15:34.725693 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.725667 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2"} err="failed to get container status \"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2\": rpc error: code = NotFound desc = could not find container \"30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2\": container with ID starting with 30448c13294cd0ef3c3a7ce5b833d215cc91e7d54c8ba4cdf8f2a428abea36d2 not found: ID does not exist" Jul 07 00:15:34.738615 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.738590 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:34.742198 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:34.742180 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2a6aa737-54dc9666bb-nqj7h"] Jul 07 00:15:35.589265 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:35.589230 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" path="/var/lib/kubelet/pods/20095afa-72bf-47ef-9b46-5c93bbaef4ed/volumes" Jul 07 00:15:44.270598 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:15:44.270561 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:17:04.174125 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:04.174085 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:17:12.579613 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:12.579578 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:17:19.050888 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.050791 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:19.051340 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.051297 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" containerName="maas-api" Jul 07 00:17:19.051340 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.051315 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" containerName="maas-api" Jul 07 00:17:19.051452 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.051419 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="20095afa-72bf-47ef-9b46-5c93bbaef4ed" containerName="maas-api" Jul 07 00:17:19.054552 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.054530 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.062687 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.062671 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4a8aca1c-serving-cert\"" Jul 07 00:17:19.063528 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.063512 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:17:19.074775 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.074725 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:19.220713 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.220676 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzgp6\" (UniqueName: \"kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.220936 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.220783 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.220936 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.220818 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.321741 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.321651 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.321741 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.321711 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzgp6\" (UniqueName: \"kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.321993 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.321812 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.321993 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:17:19.321898 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4a8aca1c-serving-cert: secret "maas-api-e2e-mt-4a8aca1c-serving-cert" not found Jul 07 00:17:19.321993 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:17:19.321961 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls podName:de9d56c1-ff31-4336-b961-c430437ae787 nodeName:}" failed. No retries permitted until 2026-07-07 00:17:19.821945496 +0000 UTC m=+1706.752002978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls") pod "maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" (UID: "de9d56c1-ff31-4336-b961-c430437ae787") : secret "maas-api-e2e-mt-4a8aca1c-serving-cert" not found Jul 07 00:17:19.322360 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.322340 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.331618 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.331584 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzgp6\" (UniqueName: \"kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.826483 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.826443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.828917 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.828897 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") pod \"maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:19.964615 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:19.964575 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:20.097595 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:20.097567 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:20.100056 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:17:20.100025 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9d56c1_ff31_4336_b961_c430437ae787.slice/crio-880cde038b9f09b75871aff61b39c64a77b8e5ac7571dfd8d4658f3e249d0aa0 WatchSource:0}: Error finding container 880cde038b9f09b75871aff61b39c64a77b8e5ac7571dfd8d4658f3e249d0aa0: Status 404 returned error can't find the container with id 880cde038b9f09b75871aff61b39c64a77b8e5ac7571dfd8d4658f3e249d0aa0 Jul 07 00:17:20.208304 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:20.208270 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" event={"ID":"de9d56c1-ff31-4336-b961-c430437ae787","Type":"ContainerStarted","Data":"880cde038b9f09b75871aff61b39c64a77b8e5ac7571dfd8d4658f3e249d0aa0"} Jul 07 00:17:21.213766 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:21.213720 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" event={"ID":"de9d56c1-ff31-4336-b961-c430437ae787","Type":"ContainerStarted","Data":"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8"} Jul 07 00:17:21.214273 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:21.213815 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:21.233387 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:21.233334 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" podStartSLOduration=1.600268945 podStartE2EDuration="2.23331427s" totalImagesPullingTime="633.045325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:21.23168994 +0000 UTC m=+1708.161747443" watchObservedRunningTime="2026-07-07 00:17:21.23331427 +0000 UTC m=+1708.163371775" Jul 07 00:17:22.471668 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:22.471627 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:17:27.223981 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:27.223948 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:30.865126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:30.865091 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:30.865578 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:30.865315 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" podUID="de9d56c1-ff31-4336-b961-c430437ae787" containerName="maas-api" containerID="cri-o://0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8" gracePeriod=30 Jul 07 00:17:31.127279 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.127206 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:31.230918 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.230862 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzgp6\" (UniqueName: \"kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6\") pod \"de9d56c1-ff31-4336-b961-c430437ae787\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " Jul 07 00:17:31.231117 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.230930 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca\") pod \"de9d56c1-ff31-4336-b961-c430437ae787\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " Jul 07 00:17:31.231117 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.231012 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") pod \"de9d56c1-ff31-4336-b961-c430437ae787\" (UID: \"de9d56c1-ff31-4336-b961-c430437ae787\") " Jul 07 00:17:31.231302 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.231280 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de9d56c1-ff31-4336-b961-c430437ae787" (UID: "de9d56c1-ff31-4336-b961-c430437ae787"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:17:31.232922 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.232899 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de9d56c1-ff31-4336-b961-c430437ae787" (UID: "de9d56c1-ff31-4336-b961-c430437ae787"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:17:31.233035 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.232931 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6" (OuterVolumeSpecName: "kube-api-access-kzgp6") pod "de9d56c1-ff31-4336-b961-c430437ae787" (UID: "de9d56c1-ff31-4336-b961-c430437ae787"). InnerVolumeSpecName "kube-api-access-kzgp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:17:31.259246 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.259215 2556 generic.go:356] "Generic (PLEG): container finished" podID="de9d56c1-ff31-4336-b961-c430437ae787" containerID="0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8" exitCode=0 Jul 07 00:17:31.259367 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.259258 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" Jul 07 00:17:31.259367 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.259293 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" event={"ID":"de9d56c1-ff31-4336-b961-c430437ae787","Type":"ContainerDied","Data":"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8"} Jul 07 00:17:31.259367 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.259358 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq" event={"ID":"de9d56c1-ff31-4336-b961-c430437ae787","Type":"ContainerDied","Data":"880cde038b9f09b75871aff61b39c64a77b8e5ac7571dfd8d4658f3e249d0aa0"} Jul 07 00:17:31.259983 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.259390 2556 scope.go:117] "RemoveContainer" containerID="0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8" Jul 07 00:17:31.268157 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.268133 2556 scope.go:117] "RemoveContainer" containerID="0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8" Jul 07 00:17:31.268453 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:17:31.268433 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8\": container with ID starting with 0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8 not found: ID does not exist" containerID="0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8" Jul 07 00:17:31.268544 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.268463 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8"} err="failed to get container status \"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8\": rpc error: code = NotFound desc = could not find container \"0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8\": container with ID starting with 0dcad4604ef1b338773e0274b6f0a0e8e60fb30de71e7a4248af13c8d9add3d8 not found: ID does not exist" Jul 07 00:17:31.286248 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.286223 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:31.290523 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.290502 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4a8aca1c-75bcf9674b-nmnxq"] Jul 07 00:17:31.332268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.332241 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzgp6\" (UniqueName: \"kubernetes.io/projected/de9d56c1-ff31-4336-b961-c430437ae787-kube-api-access-kzgp6\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:17:31.332268 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.332265 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de9d56c1-ff31-4336-b961-c430437ae787-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:17:31.332426 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.332276 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de9d56c1-ff31-4336-b961-c430437ae787-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:17:31.589035 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:31.589006 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de9d56c1-ff31-4336-b961-c430437ae787" path="/var/lib/kubelet/pods/de9d56c1-ff31-4336-b961-c430437ae787/volumes" Jul 07 00:17:35.077829 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:17:35.077778 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:18:00.386702 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.386668 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:00.387150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.387034 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de9d56c1-ff31-4336-b961-c430437ae787" containerName="maas-api" Jul 07 00:18:00.387150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.387047 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="de9d56c1-ff31-4336-b961-c430437ae787" containerName="maas-api" Jul 07 00:18:00.387150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.387107 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="de9d56c1-ff31-4336-b961-c430437ae787" containerName="maas-api" Jul 07 00:18:00.389895 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.389877 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.392668 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.392635 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1ddddfee-serving-cert\"" Jul 07 00:18:00.392829 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.392767 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:18:00.399844 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.399820 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:00.483821 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.483777 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrkd\" (UniqueName: \"kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.483987 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.483858 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.483987 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.483883 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.585285 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.585227 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrkd\" (UniqueName: \"kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.585496 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.585321 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.585496 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.585353 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.585624 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:18:00.585509 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-1ddddfee-serving-cert: secret "maas-api-e2e-mt-1ddddfee-serving-cert" not found Jul 07 00:18:00.585624 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:18:00.585579 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls podName:16dd674b-ac84-4f35-ab72-91901d52e609 nodeName:}" failed. No retries permitted until 2026-07-07 00:18:01.085556732 +0000 UTC m=+1748.015614216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls") pod "maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" (UID: "16dd674b-ac84-4f35-ab72-91901d52e609") : secret "maas-api-e2e-mt-1ddddfee-serving-cert" not found Jul 07 00:18:00.586002 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.585980 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:00.596456 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:00.596431 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrkd\" (UniqueName: \"kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:01.091041 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:01.090999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:01.093423 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:01.093401 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") pod \"maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:01.301631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:01.301596 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:01.430482 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:01.430454 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:01.432852 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:18:01.432812 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16dd674b_ac84_4f35_ab72_91901d52e609.slice/crio-e1c68ac574ba445f5dea5ef81428a02b918ca884e9866fc64e21e1f067729132 WatchSource:0}: Error finding container e1c68ac574ba445f5dea5ef81428a02b918ca884e9866fc64e21e1f067729132: Status 404 returned error can't find the container with id e1c68ac574ba445f5dea5ef81428a02b918ca884e9866fc64e21e1f067729132 Jul 07 00:18:02.410828 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:02.410787 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" event={"ID":"16dd674b-ac84-4f35-ab72-91901d52e609","Type":"ContainerStarted","Data":"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9"} Jul 07 00:18:02.411005 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:02.410854 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:02.411005 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:02.410875 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" event={"ID":"16dd674b-ac84-4f35-ab72-91901d52e609","Type":"ContainerStarted","Data":"e1c68ac574ba445f5dea5ef81428a02b918ca884e9866fc64e21e1f067729132"} Jul 07 00:18:02.428310 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:02.428247 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" podStartSLOduration=1.975024152 podStartE2EDuration="2.428231296s" totalImagesPullingTime="453.207144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:02.426657188 +0000 UTC m=+1749.356714697" watchObservedRunningTime="2026-07-07 00:18:02.428231296 +0000 UTC m=+1749.358288800" Jul 07 00:18:07.079044 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:07.079008 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:18:08.421679 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:08.421646 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:21.372051 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:21.372012 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:18:47.181361 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:47.181276 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:18:53.020562 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.020527 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:53.020998 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.020741 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" podUID="16dd674b-ac84-4f35-ab72-91901d52e609" containerName="maas-api" containerID="cri-o://270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9" gracePeriod=30 Jul 07 00:18:53.261283 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.261256 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:53.350410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.350327 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") pod \"16dd674b-ac84-4f35-ab72-91901d52e609\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " Jul 07 00:18:53.350410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.350383 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nrkd\" (UniqueName: \"kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd\") pod \"16dd674b-ac84-4f35-ab72-91901d52e609\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " Jul 07 00:18:53.350410 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.350399 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca\") pod \"16dd674b-ac84-4f35-ab72-91901d52e609\" (UID: \"16dd674b-ac84-4f35-ab72-91901d52e609\") " Jul 07 00:18:53.350720 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.350700 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16dd674b-ac84-4f35-ab72-91901d52e609" (UID: "16dd674b-ac84-4f35-ab72-91901d52e609"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:18:53.350837 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.350821 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16dd674b-ac84-4f35-ab72-91901d52e609-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:18:53.352534 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.352500 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16dd674b-ac84-4f35-ab72-91901d52e609" (UID: "16dd674b-ac84-4f35-ab72-91901d52e609"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:18:53.352627 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.352592 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd" (OuterVolumeSpecName: "kube-api-access-9nrkd") pod "16dd674b-ac84-4f35-ab72-91901d52e609" (UID: "16dd674b-ac84-4f35-ab72-91901d52e609"). InnerVolumeSpecName "kube-api-access-9nrkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:18:53.451883 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.451840 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16dd674b-ac84-4f35-ab72-91901d52e609-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:18:53.451883 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.451874 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nrkd\" (UniqueName: \"kubernetes.io/projected/16dd674b-ac84-4f35-ab72-91901d52e609-kube-api-access-9nrkd\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:18:53.611596 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.611518 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:18:53.614505 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.614484 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:18:53.614662 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.614634 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:18:53.617681 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.617660 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:18:53.642159 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.642130 2556 generic.go:356] "Generic (PLEG): container finished" podID="16dd674b-ac84-4f35-ab72-91901d52e609" containerID="270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9" exitCode=0 Jul 07 00:18:53.642311 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.642184 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" Jul 07 00:18:53.642311 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.642217 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" event={"ID":"16dd674b-ac84-4f35-ab72-91901d52e609","Type":"ContainerDied","Data":"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9"} Jul 07 00:18:53.642311 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.642273 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x" event={"ID":"16dd674b-ac84-4f35-ab72-91901d52e609","Type":"ContainerDied","Data":"e1c68ac574ba445f5dea5ef81428a02b918ca884e9866fc64e21e1f067729132"} Jul 07 00:18:53.642311 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.642297 2556 scope.go:117] "RemoveContainer" containerID="270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9" Jul 07 00:18:53.653257 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.653239 2556 scope.go:117] "RemoveContainer" containerID="270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9" Jul 07 00:18:53.653551 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:18:53.653528 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9\": container with ID starting with 270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9 not found: ID does not exist" containerID="270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9" Jul 07 00:18:53.653614 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.653562 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9"} err="failed to get container status \"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9\": rpc error: code = NotFound desc = could not find container \"270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9\": container with ID starting with 270f7381ddaf846cbef4a6fcefb9c64107770097ff5d4aa129348fe345f7e4e9 not found: ID does not exist" Jul 07 00:18:53.660004 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.659979 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:53.664738 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:53.664716 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1ddddfee-55d46fd76f-cjm2x"] Jul 07 00:18:55.588761 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:55.588711 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16dd674b-ac84-4f35-ab72-91901d52e609" path="/var/lib/kubelet/pods/16dd674b-ac84-4f35-ab72-91901d52e609/volumes" Jul 07 00:18:56.076102 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:18:56.076066 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:19:08.169535 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:19:08.169450 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:19:43.984567 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:19:43.984520 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:20:13.880351 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:20:13.880314 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:20:28.071786 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:20:28.071681 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:21:08.577569 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.577532 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:21:08.578015 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.577906 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16dd674b-ac84-4f35-ab72-91901d52e609" containerName="maas-api" Jul 07 00:21:08.578015 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.577917 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="16dd674b-ac84-4f35-ab72-91901d52e609" containerName="maas-api" Jul 07 00:21:08.578015 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.577978 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="16dd674b-ac84-4f35-ab72-91901d52e609" containerName="maas-api" Jul 07 00:21:08.581784 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.581767 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.584860 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.584839 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-b48493-serving-cert\"" Jul 07 00:21:08.585301 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.585286 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:21:08.612481 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.612459 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:21:08.728904 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.728865 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.729057 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.728913 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98hjv\" (UniqueName: \"kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.729057 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.729019 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.830041 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.829969 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.830041 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.830015 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.830209 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:21:08.830112 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-b48493-serving-cert: secret "maas-api-e2e-inf-a-b48493-serving-cert" not found Jul 07 00:21:08.830209 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:21:08.830177 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls podName:8f6eadf3-aa0c-493f-b69e-e1951af0a420 nodeName:}" failed. No retries permitted until 2026-07-07 00:21:09.330160682 +0000 UTC m=+1936.260218167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls") pod "maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" (UID: "8f6eadf3-aa0c-493f-b69e-e1951af0a420") : secret "maas-api-e2e-inf-a-b48493-serving-cert" not found Jul 07 00:21:08.830281 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.830263 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98hjv\" (UniqueName: \"kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.830613 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.830597 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:08.839700 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:08.839678 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98hjv\" (UniqueName: \"kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:09.333817 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:09.333787 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:09.336324 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:09.336299 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") pod \"maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:09.491987 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:09.491950 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:09.620510 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:09.620486 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:21:09.623210 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:21:09.623185 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6eadf3_aa0c_493f_b69e_e1951af0a420.slice/crio-52222c0a8369a02cc958d9b1c151dc3fa9e794a17bb20d306045e7e207ab7911 WatchSource:0}: Error finding container 52222c0a8369a02cc958d9b1c151dc3fa9e794a17bb20d306045e7e207ab7911: Status 404 returned error can't find the container with id 52222c0a8369a02cc958d9b1c151dc3fa9e794a17bb20d306045e7e207ab7911 Jul 07 00:21:09.624852 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:09.624825 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:21:10.250169 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:10.250135 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" event={"ID":"8f6eadf3-aa0c-493f-b69e-e1951af0a420","Type":"ContainerStarted","Data":"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579"} Jul 07 00:21:10.250361 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:10.250181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" event={"ID":"8f6eadf3-aa0c-493f-b69e-e1951af0a420","Type":"ContainerStarted","Data":"52222c0a8369a02cc958d9b1c151dc3fa9e794a17bb20d306045e7e207ab7911"} Jul 07 00:21:10.250361 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:10.250255 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:10.267987 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:10.267936 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" podStartSLOduration=1.8381109389999999 podStartE2EDuration="2.267920177s" totalImagesPullingTime="429.809238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:10.266555962 +0000 UTC m=+1937.196613467" watchObservedRunningTime="2026-07-07 00:21:10.267920177 +0000 UTC m=+1937.197977680" Jul 07 00:21:16.260295 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.260266 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:21:16.495892 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.495857 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:21:16.500130 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.500108 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.502998 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.502972 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-b8c86c-serving-cert\"" Jul 07 00:21:16.510399 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.510334 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:21:16.599538 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.599501 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.599701 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.599544 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.599701 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.599593 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj2s4\" (UniqueName: \"kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.700651 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.700618 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.700863 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:21:16.700726 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-b8c86c-serving-cert: secret "maas-api-e2e-inf-b-b8c86c-serving-cert" not found Jul 07 00:21:16.700863 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.700739 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.700863 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:21:16.700824 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls podName:b9bc70df-cb95-400b-aaa6-f0e8d0e43b39 nodeName:}" failed. No retries permitted until 2026-07-07 00:21:17.200803116 +0000 UTC m=+1944.130860617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls") pod "maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" (UID: "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39") : secret "maas-api-e2e-inf-b-b8c86c-serving-cert" not found Jul 07 00:21:16.701020 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.700892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj2s4\" (UniqueName: \"kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.701356 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.701339 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.710639 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.710610 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj2s4\" (UniqueName: \"kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:16.975895 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:16.975808 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:21:17.204169 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:17.204133 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:17.206580 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:17.206556 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") pod \"maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:17.412710 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:17.412675 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:17.548843 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:17.548804 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:21:17.552173 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:21:17.552132 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9bc70df_cb95_400b_aaa6_f0e8d0e43b39.slice/crio-a780a6cc3081a4b3c4258f89daba8b0cc5ab3245c629fdb79872c708e8b457e6 WatchSource:0}: Error finding container a780a6cc3081a4b3c4258f89daba8b0cc5ab3245c629fdb79872c708e8b457e6: Status 404 returned error can't find the container with id a780a6cc3081a4b3c4258f89daba8b0cc5ab3245c629fdb79872c708e8b457e6 Jul 07 00:21:18.288322 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:18.288290 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" event={"ID":"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39","Type":"ContainerStarted","Data":"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e"} Jul 07 00:21:18.288513 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:18.288339 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" event={"ID":"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39","Type":"ContainerStarted","Data":"a780a6cc3081a4b3c4258f89daba8b0cc5ab3245c629fdb79872c708e8b457e6"} Jul 07 00:21:18.288513 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:18.288383 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:18.306471 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:18.306418 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" podStartSLOduration=1.925381534 podStartE2EDuration="2.306403572s" totalImagesPullingTime="381.022038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:18.305537306 +0000 UTC m=+1945.235594812" watchObservedRunningTime="2026-07-07 00:21:18.306403572 +0000 UTC m=+1945.236461076" Jul 07 00:21:24.299000 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:24.298970 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:21:31.577858 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:31.577819 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:21:44.887185 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:21:44.887149 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:22:11.784764 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:11.784722 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:22:49.390926 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.390886 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:22:49.391502 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.391138 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" podUID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" containerName="maas-api" containerID="cri-o://5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579" gracePeriod=30 Jul 07 00:22:49.638377 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.638354 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:22:49.716240 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.716170 2556 generic.go:356] "Generic (PLEG): container finished" podID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" containerID="5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579" exitCode=0 Jul 07 00:22:49.716240 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.716218 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" Jul 07 00:22:49.716240 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.716228 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" event={"ID":"8f6eadf3-aa0c-493f-b69e-e1951af0a420","Type":"ContainerDied","Data":"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579"} Jul 07 00:22:49.716450 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.716267 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p" event={"ID":"8f6eadf3-aa0c-493f-b69e-e1951af0a420","Type":"ContainerDied","Data":"52222c0a8369a02cc958d9b1c151dc3fa9e794a17bb20d306045e7e207ab7911"} Jul 07 00:22:49.716450 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.716289 2556 scope.go:117] "RemoveContainer" containerID="5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579" Jul 07 00:22:49.724602 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.724587 2556 scope.go:117] "RemoveContainer" containerID="5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579" Jul 07 00:22:49.724875 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:22:49.724858 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579\": container with ID starting with 5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579 not found: ID does not exist" containerID="5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579" Jul 07 00:22:49.724931 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.724883 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579"} err="failed to get container status \"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579\": rpc error: code = NotFound desc = could not find container \"5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579\": container with ID starting with 5f8eaf46dd5aacf6421f928678faf321d723e33681cb193c90627d741fca5579 not found: ID does not exist" Jul 07 00:22:49.736677 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.736660 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca\") pod \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " Jul 07 00:22:49.736780 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.736684 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98hjv\" (UniqueName: \"kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv\") pod \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " Jul 07 00:22:49.736780 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.736710 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") pod \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\" (UID: \"8f6eadf3-aa0c-493f-b69e-e1951af0a420\") " Jul 07 00:22:49.737098 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.737068 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f6eadf3-aa0c-493f-b69e-e1951af0a420" (UID: "8f6eadf3-aa0c-493f-b69e-e1951af0a420"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:22:49.738612 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.738588 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv" (OuterVolumeSpecName: "kube-api-access-98hjv") pod "8f6eadf3-aa0c-493f-b69e-e1951af0a420" (UID: "8f6eadf3-aa0c-493f-b69e-e1951af0a420"). InnerVolumeSpecName "kube-api-access-98hjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:22:49.738711 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.738689 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f6eadf3-aa0c-493f-b69e-e1951af0a420" (UID: "8f6eadf3-aa0c-493f-b69e-e1951af0a420"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:22:49.837627 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.837599 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f6eadf3-aa0c-493f-b69e-e1951af0a420-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:22:49.837627 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.837623 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98hjv\" (UniqueName: \"kubernetes.io/projected/8f6eadf3-aa0c-493f-b69e-e1951af0a420-kube-api-access-98hjv\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:22:49.837627 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:49.837632 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f6eadf3-aa0c-493f-b69e-e1951af0a420-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:22:50.054194 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:50.054164 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:22:50.061919 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:50.061896 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b48493-d5fd4f97b-xst2p"] Jul 07 00:22:51.589098 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:51.589068 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" path="/var/lib/kubelet/pods/8f6eadf3-aa0c-493f-b69e-e1951af0a420/volumes" Jul 07 00:22:51.604095 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:22:51.604069 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:23:32.252656 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.252563 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:23:32.253203 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.252826 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" podUID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" containerName="maas-api" containerID="cri-o://cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e" gracePeriod=30 Jul 07 00:23:32.495572 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.495546 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:23:32.583521 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.583445 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") pod \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " Jul 07 00:23:32.583521 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.583489 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj2s4\" (UniqueName: \"kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4\") pod \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " Jul 07 00:23:32.583521 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.583517 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca\") pod \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\" (UID: \"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39\") " Jul 07 00:23:32.583933 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.583911 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" (UID: "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:23:32.584026 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.584003 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:23:32.585630 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.585605 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4" (OuterVolumeSpecName: "kube-api-access-tj2s4") pod "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" (UID: "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39"). InnerVolumeSpecName "kube-api-access-tj2s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:32.585630 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.585624 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" (UID: "b9bc70df-cb95-400b-aaa6-f0e8d0e43b39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:23:32.685139 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.685112 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:23:32.685139 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.685136 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj2s4\" (UniqueName: \"kubernetes.io/projected/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39-kube-api-access-tj2s4\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:23:32.907047 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.906968 2556 generic.go:356] "Generic (PLEG): container finished" podID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" containerID="cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e" exitCode=0 Jul 07 00:23:32.907047 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.907008 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" event={"ID":"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39","Type":"ContainerDied","Data":"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e"} Jul 07 00:23:32.907202 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.907069 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" event={"ID":"b9bc70df-cb95-400b-aaa6-f0e8d0e43b39","Type":"ContainerDied","Data":"a780a6cc3081a4b3c4258f89daba8b0cc5ab3245c629fdb79872c708e8b457e6"} Jul 07 00:23:32.907202 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.907091 2556 scope.go:117] "RemoveContainer" containerID="cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e" Jul 07 00:23:32.907202 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.907019 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn" Jul 07 00:23:32.916027 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.916013 2556 scope.go:117] "RemoveContainer" containerID="cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e" Jul 07 00:23:32.916283 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:23:32.916263 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e\": container with ID starting with cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e not found: ID does not exist" containerID="cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e" Jul 07 00:23:32.916356 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.916291 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e"} err="failed to get container status \"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e\": rpc error: code = NotFound desc = could not find container \"cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e\": container with ID starting with cbd28710802b14b3169b15eeccc67fdfd4add33793b6d6c52073acb86465807e not found: ID does not exist" Jul 07 00:23:32.934767 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.934724 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:23:32.940538 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:32.940521 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-b8c86c-6dd7b54649-z74hn"] Jul 07 00:23:33.589606 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:33.589577 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" path="/var/lib/kubelet/pods/b9bc70df-cb95-400b-aaa6-f0e8d0e43b39/volumes" Jul 07 00:23:35.471829 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:35.471796 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:23:42.595254 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595216 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:23:42.595785 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595766 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" containerName="maas-api" Jul 07 00:23:42.595844 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595789 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" containerName="maas-api" Jul 07 00:23:42.595844 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595801 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" containerName="maas-api" Jul 07 00:23:42.595844 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595810 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" containerName="maas-api" Jul 07 00:23:42.595946 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595909 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f6eadf3-aa0c-493f-b69e-e1951af0a420" containerName="maas-api" Jul 07 00:23:42.595946 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.595922 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9bc70df-cb95-400b-aaa6-f0e8d0e43b39" containerName="maas-api" Jul 07 00:23:42.600623 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.600601 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.603558 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.603537 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-2833a6-serving-cert\"" Jul 07 00:23:42.603719 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.603704 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:23:42.610397 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.610374 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:23:42.770697 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.770663 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.770899 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.770770 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.770899 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.770798 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppsx6\" (UniqueName: \"kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.872021 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.871952 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.872021 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.871986 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppsx6\" (UniqueName: \"kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.872021 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.872011 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.872270 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:23:42.872111 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-2833a6-serving-cert: secret "maas-api-e2e-api-a-2833a6-serving-cert" not found Jul 07 00:23:42.872270 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:23:42.872195 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls podName:c8525e94-0a9a-493c-89a1-78b6e17490bd nodeName:}" failed. No retries permitted until 2026-07-07 00:23:43.372173312 +0000 UTC m=+2090.302230795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls") pod "maas-api-e2e-api-a-2833a6-798d7499c6-snk77" (UID: "c8525e94-0a9a-493c-89a1-78b6e17490bd") : secret "maas-api-e2e-api-a-2833a6-serving-cert" not found Jul 07 00:23:42.872604 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.872587 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:42.886896 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:42.886875 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppsx6\" (UniqueName: \"kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:43.376385 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:43.376349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:43.378858 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:43.378830 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") pod \"maas-api-e2e-api-a-2833a6-798d7499c6-snk77\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:43.511369 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:43.511337 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:43.869363 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:43.869332 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:23:43.872405 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:23:43.872374 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8525e94_0a9a_493c_89a1_78b6e17490bd.slice/crio-ef3f7cd388a77632e2531c3e3cfd71e2bed3b1667d80754fe2dc45525d232750 WatchSource:0}: Error finding container ef3f7cd388a77632e2531c3e3cfd71e2bed3b1667d80754fe2dc45525d232750: Status 404 returned error can't find the container with id ef3f7cd388a77632e2531c3e3cfd71e2bed3b1667d80754fe2dc45525d232750 Jul 07 00:23:43.968653 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:43.968615 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" event={"ID":"c8525e94-0a9a-493c-89a1-78b6e17490bd","Type":"ContainerStarted","Data":"ef3f7cd388a77632e2531c3e3cfd71e2bed3b1667d80754fe2dc45525d232750"} Jul 07 00:23:44.974436 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:44.974405 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" event={"ID":"c8525e94-0a9a-493c-89a1-78b6e17490bd","Type":"ContainerStarted","Data":"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74"} Jul 07 00:23:44.974829 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:44.974538 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:44.992284 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:44.992241 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" podStartSLOduration=2.437224753 podStartE2EDuration="2.992227053s" totalImagesPullingTime="555.0023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:44.990888405 +0000 UTC m=+2091.920945910" watchObservedRunningTime="2026-07-07 00:23:44.992227053 +0000 UTC m=+2091.922284557" Jul 07 00:23:46.278953 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:46.278913 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:23:50.608485 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.608452 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:23:50.612397 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.612372 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.642042 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.642014 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.642150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.642050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbqmc\" (UniqueName: \"kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.642150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.642108 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.642816 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.642800 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-28187d-serving-cert\"" Jul 07 00:23:50.664084 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.664050 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:23:50.743235 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.743202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.743418 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.743243 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbqmc\" (UniqueName: \"kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.743418 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.743289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.744009 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.743984 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.745798 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.745779 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.754029 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.754009 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbqmc\" (UniqueName: \"kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc\") pod \"maas-api-e2e-api-b-28187d-785b9d748-h9v4h\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.923060 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.922981 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:50.987477 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:50.986701 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:23:51.072633 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:51.072601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:23:51.075529 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:23:51.075491 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ce09a7_2a22_42f2_91a0_878766e6e665.slice/crio-73c3f152f77ffd05899fc8653281c5ce276a1aedfcd088cf76da572b28e6db5a WatchSource:0}: Error finding container 73c3f152f77ffd05899fc8653281c5ce276a1aedfcd088cf76da572b28e6db5a: Status 404 returned error can't find the container with id 73c3f152f77ffd05899fc8653281c5ce276a1aedfcd088cf76da572b28e6db5a Jul 07 00:23:52.009173 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:52.009132 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" event={"ID":"f4ce09a7-2a22-42f2-91a0-878766e6e665","Type":"ContainerStarted","Data":"0ff34ad6b74187a90f24c937c51c22f64ca1df765b1447e9a9d41e07abaad0b9"} Jul 07 00:23:52.009631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:52.009187 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" event={"ID":"f4ce09a7-2a22-42f2-91a0-878766e6e665","Type":"ContainerStarted","Data":"73c3f152f77ffd05899fc8653281c5ce276a1aedfcd088cf76da572b28e6db5a"} Jul 07 00:23:52.009631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:52.009274 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:23:52.027305 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:52.027259 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" podStartSLOduration=1.658283105 podStartE2EDuration="2.027245116s" totalImagesPullingTime="368.962011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:52.025508566 +0000 UTC m=+2098.955566072" watchObservedRunningTime="2026-07-07 00:23:52.027245116 +0000 UTC m=+2098.957302620" Jul 07 00:23:53.639414 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:53.639383 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:23:53.642117 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:53.642085 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:23:53.643290 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:53.643271 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:23:53.646324 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:53.646309 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:23:58.018873 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:23:58.018846 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:24:03.918346 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:03.918312 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:24:04.310327 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.310296 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:24:04.310551 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.310529 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" podUID="c8525e94-0a9a-493c-89a1-78b6e17490bd" containerName="maas-api" containerID="cri-o://492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74" gracePeriod=30 Jul 07 00:24:04.558370 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.558347 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:24:04.655009 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.654942 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") pod \"c8525e94-0a9a-493c-89a1-78b6e17490bd\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " Jul 07 00:24:04.655009 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.655004 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppsx6\" (UniqueName: \"kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6\") pod \"c8525e94-0a9a-493c-89a1-78b6e17490bd\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " Jul 07 00:24:04.655176 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.655028 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca\") pod \"c8525e94-0a9a-493c-89a1-78b6e17490bd\" (UID: \"c8525e94-0a9a-493c-89a1-78b6e17490bd\") " Jul 07 00:24:04.655398 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.655375 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8525e94-0a9a-493c-89a1-78b6e17490bd" (UID: "c8525e94-0a9a-493c-89a1-78b6e17490bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:04.657064 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.657041 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6" (OuterVolumeSpecName: "kube-api-access-ppsx6") pod "c8525e94-0a9a-493c-89a1-78b6e17490bd" (UID: "c8525e94-0a9a-493c-89a1-78b6e17490bd"). InnerVolumeSpecName "kube-api-access-ppsx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:04.657189 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.657081 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8525e94-0a9a-493c-89a1-78b6e17490bd" (UID: "c8525e94-0a9a-493c-89a1-78b6e17490bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:24:04.755888 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.755864 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8525e94-0a9a-493c-89a1-78b6e17490bd-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:04.755888 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.755885 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppsx6\" (UniqueName: \"kubernetes.io/projected/c8525e94-0a9a-493c-89a1-78b6e17490bd-kube-api-access-ppsx6\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:04.756032 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:04.755895 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8525e94-0a9a-493c-89a1-78b6e17490bd-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:05.068235 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.068207 2556 generic.go:356] "Generic (PLEG): container finished" podID="c8525e94-0a9a-493c-89a1-78b6e17490bd" containerID="492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74" exitCode=0 Jul 07 00:24:05.068631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.068253 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" Jul 07 00:24:05.068631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.068269 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" event={"ID":"c8525e94-0a9a-493c-89a1-78b6e17490bd","Type":"ContainerDied","Data":"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74"} Jul 07 00:24:05.068631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.068307 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77" event={"ID":"c8525e94-0a9a-493c-89a1-78b6e17490bd","Type":"ContainerDied","Data":"ef3f7cd388a77632e2531c3e3cfd71e2bed3b1667d80754fe2dc45525d232750"} Jul 07 00:24:05.068631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.068330 2556 scope.go:117] "RemoveContainer" containerID="492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74" Jul 07 00:24:05.077797 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.077779 2556 scope.go:117] "RemoveContainer" containerID="492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74" Jul 07 00:24:05.078076 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:24:05.078056 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74\": container with ID starting with 492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74 not found: ID does not exist" containerID="492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74" Jul 07 00:24:05.078154 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.078087 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74"} err="failed to get container status \"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74\": rpc error: code = NotFound desc = could not find container \"492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74\": container with ID starting with 492b98a4844a5f5e09ed3c0ebf9162794e01c0865094046b71fc9536a1e10f74 not found: ID does not exist" Jul 07 00:24:05.091674 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.091652 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:24:05.096550 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.096530 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2833a6-798d7499c6-snk77"] Jul 07 00:24:05.589545 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:05.589513 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8525e94-0a9a-493c-89a1-78b6e17490bd" path="/var/lib/kubelet/pods/c8525e94-0a9a-493c-89a1-78b6e17490bd/volumes" Jul 07 00:24:10.935859 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:10.935828 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:24:10.936326 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:10.936057 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" podUID="f4ce09a7-2a22-42f2-91a0-878766e6e665" containerName="maas-api" containerID="cri-o://0ff34ad6b74187a90f24c937c51c22f64ca1df765b1447e9a9d41e07abaad0b9" gracePeriod=30 Jul 07 00:24:11.098103 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.098067 2556 generic.go:356] "Generic (PLEG): container finished" podID="f4ce09a7-2a22-42f2-91a0-878766e6e665" containerID="0ff34ad6b74187a90f24c937c51c22f64ca1df765b1447e9a9d41e07abaad0b9" exitCode=0 Jul 07 00:24:11.098257 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.098124 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" event={"ID":"f4ce09a7-2a22-42f2-91a0-878766e6e665","Type":"ContainerDied","Data":"0ff34ad6b74187a90f24c937c51c22f64ca1df765b1447e9a9d41e07abaad0b9"} Jul 07 00:24:11.178478 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.178450 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:24:11.207486 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.207418 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca\") pod \"f4ce09a7-2a22-42f2-91a0-878766e6e665\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " Jul 07 00:24:11.207486 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.207467 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbqmc\" (UniqueName: \"kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc\") pod \"f4ce09a7-2a22-42f2-91a0-878766e6e665\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " Jul 07 00:24:11.207679 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.207503 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls\") pod \"f4ce09a7-2a22-42f2-91a0-878766e6e665\" (UID: \"f4ce09a7-2a22-42f2-91a0-878766e6e665\") " Jul 07 00:24:11.207827 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.207803 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4ce09a7-2a22-42f2-91a0-878766e6e665" (UID: "f4ce09a7-2a22-42f2-91a0-878766e6e665"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:11.209556 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.209525 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4ce09a7-2a22-42f2-91a0-878766e6e665" (UID: "f4ce09a7-2a22-42f2-91a0-878766e6e665"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:24:11.209556 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.209549 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc" (OuterVolumeSpecName: "kube-api-access-tbqmc") pod "f4ce09a7-2a22-42f2-91a0-878766e6e665" (UID: "f4ce09a7-2a22-42f2-91a0-878766e6e665"). InnerVolumeSpecName "kube-api-access-tbqmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:11.308149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.308110 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4ce09a7-2a22-42f2-91a0-878766e6e665-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:11.308149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.308143 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4ce09a7-2a22-42f2-91a0-878766e6e665-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:11.308149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:11.308153 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbqmc\" (UniqueName: \"kubernetes.io/projected/f4ce09a7-2a22-42f2-91a0-878766e6e665-kube-api-access-tbqmc\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:24:12.104567 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:12.104535 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" event={"ID":"f4ce09a7-2a22-42f2-91a0-878766e6e665","Type":"ContainerDied","Data":"73c3f152f77ffd05899fc8653281c5ce276a1aedfcd088cf76da572b28e6db5a"} Jul 07 00:24:12.104567 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:12.104555 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h" Jul 07 00:24:12.105056 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:12.104593 2556 scope.go:117] "RemoveContainer" containerID="0ff34ad6b74187a90f24c937c51c22f64ca1df765b1447e9a9d41e07abaad0b9" Jul 07 00:24:12.121998 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:12.121976 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:24:12.125311 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:12.125292 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-28187d-785b9d748-h9v4h"] Jul 07 00:24:13.590385 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:13.590352 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ce09a7-2a22-42f2-91a0-878766e6e665" path="/var/lib/kubelet/pods/f4ce09a7-2a22-42f2-91a0-878766e6e665/volumes" Jul 07 00:24:15.073657 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:15.073623 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:24:20.770571 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770539 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:24:20.770958 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770890 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ce09a7-2a22-42f2-91a0-878766e6e665" containerName="maas-api" Jul 07 00:24:20.770958 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770901 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ce09a7-2a22-42f2-91a0-878766e6e665" containerName="maas-api" Jul 07 00:24:20.770958 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770919 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8525e94-0a9a-493c-89a1-78b6e17490bd" containerName="maas-api" Jul 07 00:24:20.770958 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770925 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8525e94-0a9a-493c-89a1-78b6e17490bd" containerName="maas-api" Jul 07 00:24:20.771084 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770974 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8525e94-0a9a-493c-89a1-78b6e17490bd" containerName="maas-api" Jul 07 00:24:20.771084 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.770985 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ce09a7-2a22-42f2-91a0-878766e6e665" containerName="maas-api" Jul 07 00:24:20.775389 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.775372 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.778045 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.778016 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-fced7e-serving-cert\"" Jul 07 00:24:20.778199 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.778094 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 07 00:24:20.785878 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.785851 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:24:20.888246 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.888217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.888386 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.888257 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.888386 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.888341 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmfvv\" (UniqueName: \"kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.989401 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.989375 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmfvv\" (UniqueName: \"kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.989565 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.989438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.989565 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.989468 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.990214 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.990190 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.991807 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.991791 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:20.997588 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:20.997565 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmfvv\" (UniqueName: \"kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv\") pod \"maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:21.087094 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:21.087039 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:21.222703 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:21.222679 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:24:21.225053 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:24:21.225026 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7535436_5ff0_4266_a155_fa45417a28d4.slice/crio-d8e2db0e68b14fb7c73735713932a69a4aaab4ed4815760de8b8930e20ce9e88 WatchSource:0}: Error finding container d8e2db0e68b14fb7c73735713932a69a4aaab4ed4815760de8b8930e20ce9e88: Status 404 returned error can't find the container with id d8e2db0e68b14fb7c73735713932a69a4aaab4ed4815760de8b8930e20ce9e88 Jul 07 00:24:22.152782 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:22.152732 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" event={"ID":"d7535436-5ff0-4266-a155-fa45417a28d4","Type":"ContainerStarted","Data":"36a44623ee3fbeee788e830168054c9900699ce30d0024b057734a6509b53576"} Jul 07 00:24:22.153235 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:22.152799 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" event={"ID":"d7535436-5ff0-4266-a155-fa45417a28d4","Type":"ContainerStarted","Data":"d8e2db0e68b14fb7c73735713932a69a4aaab4ed4815760de8b8930e20ce9e88"} Jul 07 00:24:22.153235 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:22.152909 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:22.171279 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:22.171237 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" podStartSLOduration=1.750209948 podStartE2EDuration="2.171223464s" totalImagesPullingTime="421.013516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:22.170236764 +0000 UTC m=+2129.100294271" watchObservedRunningTime="2026-07-07 00:24:22.171223464 +0000 UTC m=+2129.101280968" Jul 07 00:24:26.181846 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:26.181812 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:24:28.163005 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.162979 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:24:28.810946 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.810909 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:24:28.814595 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.814574 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:28.817416 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.817395 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-accfb9-serving-cert\"" Jul 07 00:24:28.835302 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.835278 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:24:28.965162 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.965124 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:28.965390 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.965174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:28.965390 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:28.965285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz4fm\" (UniqueName: \"kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.066727 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.066649 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.066727 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.066687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.066727 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.066719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz4fm\" (UniqueName: \"kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.067367 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.067343 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.068994 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.068972 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.075151 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.075122 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz4fm\" (UniqueName: \"kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm\") pod \"maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.125078 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.125046 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:29.260734 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:29.260687 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:24:29.263582 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:24:29.263556 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58014652_e13e_401d_9597_bbefc798c82f.slice/crio-481047fa3a37782b3cbc9a8987d6829c1b31b2119e211c320db1843c7eb5c06c WatchSource:0}: Error finding container 481047fa3a37782b3cbc9a8987d6829c1b31b2119e211c320db1843c7eb5c06c: Status 404 returned error can't find the container with id 481047fa3a37782b3cbc9a8987d6829c1b31b2119e211c320db1843c7eb5c06c Jul 07 00:24:30.194022 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:30.193994 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" event={"ID":"58014652-e13e-401d-9597-bbefc798c82f","Type":"ContainerStarted","Data":"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a"} Jul 07 00:24:30.194187 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:30.194046 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" event={"ID":"58014652-e13e-401d-9597-bbefc798c82f","Type":"ContainerStarted","Data":"481047fa3a37782b3cbc9a8987d6829c1b31b2119e211c320db1843c7eb5c06c"} Jul 07 00:24:30.194187 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:30.194123 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:30.211180 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:30.211134 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" podStartSLOduration=1.741364092 podStartE2EDuration="2.21112102s" totalImagesPullingTime="469.756928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:30.20967725 +0000 UTC m=+2137.139734755" watchObservedRunningTime="2026-07-07 00:24:30.21112102 +0000 UTC m=+2137.141178576" Jul 07 00:24:36.203512 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:36.203484 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:24:44.180907 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:24:44.180862 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:25:16.167785 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:25:16.167738 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:25:29.070946 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:25:29.070913 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:25:45.370667 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:25:45.370631 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:26:01.677060 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:26:01.677024 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:26:15.669894 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:26:15.669802 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:26:33.779680 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:26:33.779645 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:26:50.068528 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:26:50.068494 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:27:07.777141 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:27:07.777100 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:27:22.178121 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:27:22.178082 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:27:57.185378 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:27:57.185297 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:28:21.470078 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:21.470038 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:28:39.582727 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:39.582691 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:28:53.667051 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:53.667023 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:28:53.670160 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:53.670135 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:28:53.672262 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:53.672243 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:28:53.675407 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:53.675385 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4b9rn_327e2de0-9591-458f-9728-79adfc2c46eb/ovn-acl-logging/0.log" Jul 07 00:28:57.483635 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:28:57.483601 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:29:15.844358 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:15.844276 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:29:29.781302 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:29.781267 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:29:44.382935 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:44.382902 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:29:53.437739 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.437703 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:29:53.438370 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.438031 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" podUID="d7535436-5ff0-4266-a155-fa45417a28d4" containerName="maas-api" containerID="cri-o://36a44623ee3fbeee788e830168054c9900699ce30d0024b057734a6509b53576" gracePeriod=30 Jul 07 00:29:53.666044 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.665993 2556 generic.go:356] "Generic (PLEG): container finished" podID="d7535436-5ff0-4266-a155-fa45417a28d4" containerID="36a44623ee3fbeee788e830168054c9900699ce30d0024b057734a6509b53576" exitCode=0 Jul 07 00:29:53.666122 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.666052 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" event={"ID":"d7535436-5ff0-4266-a155-fa45417a28d4","Type":"ContainerDied","Data":"36a44623ee3fbeee788e830168054c9900699ce30d0024b057734a6509b53576"} Jul 07 00:29:53.685467 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.685443 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:29:53.820161 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.820132 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca\") pod \"d7535436-5ff0-4266-a155-fa45417a28d4\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " Jul 07 00:29:53.820363 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.820238 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmfvv\" (UniqueName: \"kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv\") pod \"d7535436-5ff0-4266-a155-fa45417a28d4\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " Jul 07 00:29:53.820363 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.820286 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls\") pod \"d7535436-5ff0-4266-a155-fa45417a28d4\" (UID: \"d7535436-5ff0-4266-a155-fa45417a28d4\") " Jul 07 00:29:53.820512 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.820488 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7535436-5ff0-4266-a155-fa45417a28d4" (UID: "d7535436-5ff0-4266-a155-fa45417a28d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:29:53.822369 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.822341 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv" (OuterVolumeSpecName: "kube-api-access-cmfvv") pod "d7535436-5ff0-4266-a155-fa45417a28d4" (UID: "d7535436-5ff0-4266-a155-fa45417a28d4"). InnerVolumeSpecName "kube-api-access-cmfvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:29:53.822479 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.822417 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7535436-5ff0-4266-a155-fa45417a28d4" (UID: "d7535436-5ff0-4266-a155-fa45417a28d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:29:53.921886 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.921853 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7535436-5ff0-4266-a155-fa45417a28d4-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:29:53.921886 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.921883 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535436-5ff0-4266-a155-fa45417a28d4-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:29:53.922138 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:53.921896 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmfvv\" (UniqueName: \"kubernetes.io/projected/d7535436-5ff0-4266-a155-fa45417a28d4-kube-api-access-cmfvv\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:29:54.671720 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:54.671686 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" Jul 07 00:29:54.671720 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:54.671709 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg" event={"ID":"d7535436-5ff0-4266-a155-fa45417a28d4","Type":"ContainerDied","Data":"d8e2db0e68b14fb7c73735713932a69a4aaab4ed4815760de8b8930e20ce9e88"} Jul 07 00:29:54.672182 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:54.671785 2556 scope.go:117] "RemoveContainer" containerID="36a44623ee3fbeee788e830168054c9900699ce30d0024b057734a6509b53576" Jul 07 00:29:54.693927 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:54.693902 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:29:54.697394 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:54.697373 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-fced7e-c5866f546-gjkvg"] Jul 07 00:29:55.589835 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:29:55.589803 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7535436-5ff0-4266-a155-fa45417a28d4" path="/var/lib/kubelet/pods/d7535436-5ff0-4266-a155-fa45417a28d4/volumes" Jul 07 00:30:00.132357 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.132318 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-nlx56"] Jul 07 00:30:00.132863 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.132657 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7535436-5ff0-4266-a155-fa45417a28d4" containerName="maas-api" Jul 07 00:30:00.132863 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.132667 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7535436-5ff0-4266-a155-fa45417a28d4" containerName="maas-api" Jul 07 00:30:00.132863 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.132736 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7535436-5ff0-4266-a155-fa45417a28d4" containerName="maas-api" Jul 07 00:30:00.135732 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.135713 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:00.135898 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.135876 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v"] Jul 07 00:30:00.139190 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.139174 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:00.146840 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.146820 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-nlx56"] Jul 07 00:30:00.149274 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.149248 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v"] Jul 07 00:30:00.275952 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.275917 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5zbx\" (UniqueName: \"kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx\") pod \"maas-api-key-cleanup-29723070-nlx56\" (UID: \"8697f2e8-f8c7-463d-b86e-9fe05069fbcd\") " pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:00.276148 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.275971 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnbhd\" (UniqueName: \"kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd\") pod \"maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v\" (UID: \"7280b3c2-0395-438a-b81f-6c8064fe4b17\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:00.377251 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.377212 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5zbx\" (UniqueName: \"kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx\") pod \"maas-api-key-cleanup-29723070-nlx56\" (UID: \"8697f2e8-f8c7-463d-b86e-9fe05069fbcd\") " pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:00.377453 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.377266 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnbhd\" (UniqueName: \"kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd\") pod \"maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v\" (UID: \"7280b3c2-0395-438a-b81f-6c8064fe4b17\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:00.385581 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.385506 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5zbx\" (UniqueName: \"kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx\") pod \"maas-api-key-cleanup-29723070-nlx56\" (UID: \"8697f2e8-f8c7-463d-b86e-9fe05069fbcd\") " pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:00.385712 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.385599 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnbhd\" (UniqueName: \"kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd\") pod \"maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v\" (UID: \"7280b3c2-0395-438a-b81f-6c8064fe4b17\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:00.447709 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.447662 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:00.454975 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.454951 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:00.592620 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.592592 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v"] Jul 07 00:30:00.594191 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:30:00.594165 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7280b3c2_0395_438a_b81f_6c8064fe4b17.slice/crio-0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f WatchSource:0}: Error finding container 0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f: Status 404 returned error can't find the container with id 0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f Jul 07 00:30:00.609928 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.609901 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-nlx56"] Jul 07 00:30:00.612179 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:30:00.612154 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8697f2e8_f8c7_463d_b86e_9fe05069fbcd.slice/crio-3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e WatchSource:0}: Error finding container 3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e: Status 404 returned error can't find the container with id 3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e Jul 07 00:30:00.700893 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.700790 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" event={"ID":"7280b3c2-0395-438a-b81f-6c8064fe4b17","Type":"ContainerStarted","Data":"0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f"} Jul 07 00:30:00.702246 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:00.702218 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" event={"ID":"8697f2e8-f8c7-463d-b86e-9fe05069fbcd","Type":"ContainerStarted","Data":"3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e"} Jul 07 00:30:01.708129 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:01.708093 2556 generic.go:356] "Generic (PLEG): container finished" podID="7280b3c2-0395-438a-b81f-6c8064fe4b17" containerID="9a5ae945f67a7175fe442c449965a7e55e90d47ae30548c6e4e3ac85977f65aa" exitCode=0 Jul 07 00:30:01.708542 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:01.708173 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" event={"ID":"7280b3c2-0395-438a-b81f-6c8064fe4b17","Type":"ContainerDied","Data":"9a5ae945f67a7175fe442c449965a7e55e90d47ae30548c6e4e3ac85977f65aa"} Jul 07 00:30:01.709665 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:01.709643 2556 generic.go:356] "Generic (PLEG): container finished" podID="8697f2e8-f8c7-463d-b86e-9fe05069fbcd" containerID="86bb923019bb03c20de851cfc4d109d27a729216e02e4f5d099c340534a8f330" exitCode=0 Jul 07 00:30:01.709802 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:01.709690 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" event={"ID":"8697f2e8-f8c7-463d-b86e-9fe05069fbcd","Type":"ContainerDied","Data":"86bb923019bb03c20de851cfc4d109d27a729216e02e4f5d099c340534a8f330"} Jul 07 00:30:01.778348 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:01.778248 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:30:02.861739 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:02.861719 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:02.880366 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:02.880338 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:03.004249 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.004212 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5zbx\" (UniqueName: \"kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx\") pod \"8697f2e8-f8c7-463d-b86e-9fe05069fbcd\" (UID: \"8697f2e8-f8c7-463d-b86e-9fe05069fbcd\") " Jul 07 00:30:03.004249 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.004255 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnbhd\" (UniqueName: \"kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd\") pod \"7280b3c2-0395-438a-b81f-6c8064fe4b17\" (UID: \"7280b3c2-0395-438a-b81f-6c8064fe4b17\") " Jul 07 00:30:03.006459 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.006425 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd" (OuterVolumeSpecName: "kube-api-access-jnbhd") pod "7280b3c2-0395-438a-b81f-6c8064fe4b17" (UID: "7280b3c2-0395-438a-b81f-6c8064fe4b17"). InnerVolumeSpecName "kube-api-access-jnbhd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:03.006459 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.006434 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx" (OuterVolumeSpecName: "kube-api-access-b5zbx") pod "8697f2e8-f8c7-463d-b86e-9fe05069fbcd" (UID: "8697f2e8-f8c7-463d-b86e-9fe05069fbcd"). InnerVolumeSpecName "kube-api-access-b5zbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:03.105423 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.105377 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5zbx\" (UniqueName: \"kubernetes.io/projected/8697f2e8-f8c7-463d-b86e-9fe05069fbcd-kube-api-access-b5zbx\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:30:03.105423 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.105415 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnbhd\" (UniqueName: \"kubernetes.io/projected/7280b3c2-0395-438a-b81f-6c8064fe4b17-kube-api-access-jnbhd\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:30:03.720414 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.720384 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" Jul 07 00:30:03.720609 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.720425 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-nlx56" event={"ID":"8697f2e8-f8c7-463d-b86e-9fe05069fbcd","Type":"ContainerDied","Data":"3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e"} Jul 07 00:30:03.720609 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.720475 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b942f97a2343391b01fb2c9c82d7a04c72b95a7ea6229637656fb978fcfbe7e" Jul 07 00:30:03.721781 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.721734 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" event={"ID":"7280b3c2-0395-438a-b81f-6c8064fe4b17","Type":"ContainerDied","Data":"0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f"} Jul 07 00:30:03.721929 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.721789 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v" Jul 07 00:30:03.721929 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:03.721794 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0570f410304262249dfbb5f61806fa58b9e547aaaa67d475ea5773cea3c5604f" Jul 07 00:30:36.534204 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.534168 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:30:36.535187 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.534422 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" podUID="58014652-e13e-401d-9597-bbefc798c82f" containerName="maas-api" containerID="cri-o://32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a" gracePeriod=30 Jul 07 00:30:36.560146 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.560106 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v"] Jul 07 00:30:36.561630 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.561607 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-accfb9-29723070-9pr7v"] Jul 07 00:30:36.780056 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.780031 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:30:36.871158 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.871065 2556 generic.go:356] "Generic (PLEG): container finished" podID="58014652-e13e-401d-9597-bbefc798c82f" containerID="32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a" exitCode=0 Jul 07 00:30:36.871158 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.871110 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" Jul 07 00:30:36.871158 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.871141 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" event={"ID":"58014652-e13e-401d-9597-bbefc798c82f","Type":"ContainerDied","Data":"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a"} Jul 07 00:30:36.871396 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.871192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms" event={"ID":"58014652-e13e-401d-9597-bbefc798c82f","Type":"ContainerDied","Data":"481047fa3a37782b3cbc9a8987d6829c1b31b2119e211c320db1843c7eb5c06c"} Jul 07 00:30:36.871396 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.871215 2556 scope.go:117] "RemoveContainer" containerID="32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a" Jul 07 00:30:36.879479 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.879463 2556 scope.go:117] "RemoveContainer" containerID="32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a" Jul 07 00:30:36.879773 ip-10-0-132-169 kubenswrapper[2556]: E0707 00:30:36.879737 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a\": container with ID starting with 32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a not found: ID does not exist" containerID="32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a" Jul 07 00:30:36.879847 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.879780 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a"} err="failed to get container status \"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a\": rpc error: code = NotFound desc = could not find container \"32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a\": container with ID starting with 32b19aff190d12055725ced043a9ce75901613247d45e2bb04d29bf7de75cf8a not found: ID does not exist" Jul 07 00:30:36.903093 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.903058 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca\") pod \"58014652-e13e-401d-9597-bbefc798c82f\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " Jul 07 00:30:36.903234 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.903105 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz4fm\" (UniqueName: \"kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm\") pod \"58014652-e13e-401d-9597-bbefc798c82f\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " Jul 07 00:30:36.903283 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.903233 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls\") pod \"58014652-e13e-401d-9597-bbefc798c82f\" (UID: \"58014652-e13e-401d-9597-bbefc798c82f\") " Jul 07 00:30:36.903502 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.903469 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58014652-e13e-401d-9597-bbefc798c82f" (UID: "58014652-e13e-401d-9597-bbefc798c82f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:30:36.905176 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.905154 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58014652-e13e-401d-9597-bbefc798c82f" (UID: "58014652-e13e-401d-9597-bbefc798c82f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:30:36.905287 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:36.905269 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm" (OuterVolumeSpecName: "kube-api-access-dz4fm") pod "58014652-e13e-401d-9597-bbefc798c82f" (UID: "58014652-e13e-401d-9597-bbefc798c82f"). InnerVolumeSpecName "kube-api-access-dz4fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:37.004345 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.004299 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58014652-e13e-401d-9597-bbefc798c82f-openshift-service-ca\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:30:37.004345 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.004336 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz4fm\" (UniqueName: \"kubernetes.io/projected/58014652-e13e-401d-9597-bbefc798c82f-kube-api-access-dz4fm\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:30:37.004345 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.004347 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58014652-e13e-401d-9597-bbefc798c82f-maas-api-tls\") on node \"ip-10-0-132-169.ec2.internal\" DevicePath \"\"" Jul 07 00:30:37.209056 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.209017 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:30:37.215135 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.215107 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-accfb9-98f7f897c-2fcms"] Jul 07 00:30:37.589766 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.589712 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58014652-e13e-401d-9597-bbefc798c82f" path="/var/lib/kubelet/pods/58014652-e13e-401d-9597-bbefc798c82f/volumes" Jul 07 00:30:37.590260 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:37.590212 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7280b3c2-0395-438a-b81f-6c8064fe4b17" path="/var/lib/kubelet/pods/7280b3c2-0395-438a-b81f-6c8064fe4b17/volumes" Jul 07 00:30:39.373561 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:39.373520 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8bflg"] Jul 07 00:30:48.544081 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:48.544003 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-rbf8n_30826312-2250-4686-9100-868b4b1ff9c3/cleanup/0.log" Jul 07 00:30:48.660672 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:48.660632 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723070-nlx56_8697f2e8-f8c7-463d-b86e-9fe05069fbcd/cleanup/0.log" Jul 07 00:30:49.157775 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:49.157729 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-55bbr_b095a823-55df-482e-81a6-6ba7315876da/copy-manifests/0.log" Jul 07 00:30:49.165242 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:49.165218 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-55bbr_b095a823-55df-482e-81a6-6ba7315876da/manager/0.log" Jul 07 00:30:51.669712 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:51.669681 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8bflg_66c18bcd-10ed-4637-b833-28d72a81a9af/limitador/0.log" Jul 07 00:30:52.312463 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:52.312431 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-99ckj_462f541c-54a8-41d2-abf1-fbcb1df4e80c/discovery/0.log" Jul 07 00:30:53.062161 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:53.062134 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9b6988657-sb8wz_c1bfd36e-da1a-49d8-859e-1c66d4bc3e98/router/0.log" Jul 07 00:30:53.795150 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:30:53.795125 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6968h_13848a88-dc5c-46b8-b00c-eaa0d5d1c76c/main/0.log" Jul 07 00:31:01.708674 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:01.708649 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ckczt_283f81b9-52b3-482e-9f4b-6bf4cf1c09f2/global-pull-secret-syncer/0.log" Jul 07 00:31:01.840089 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:01.840062 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h9g4p_c4692220-7d1f-4e3d-9dbd-3485cb55e704/konnectivity-agent/0.log" Jul 07 00:31:01.918262 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:01.918238 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-169.ec2.internal_12306cecd9bb065dc086ef079ccc1976/haproxy/0.log" Jul 07 00:31:06.897677 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:06.897597 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8bflg_66c18bcd-10ed-4637-b833-28d72a81a9af/limitador/0.log" Jul 07 00:31:09.070911 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.070882 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gx7vd_5e6dd046-0cd6-4758-9676-10a77836b25b/node-exporter/0.log" Jul 07 00:31:09.095205 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.095146 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gx7vd_5e6dd046-0cd6-4758-9676-10a77836b25b/kube-rbac-proxy/0.log" Jul 07 00:31:09.115644 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.115623 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gx7vd_5e6dd046-0cd6-4758-9676-10a77836b25b/init-textfile/0.log" Jul 07 00:31:09.231883 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.231850 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x5gks_4e40c58d-524f-4a93-9947-8296c8e84cbc/kube-rbac-proxy-main/0.log" Jul 07 00:31:09.256995 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.256971 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x5gks_4e40c58d-524f-4a93-9947-8296c8e84cbc/kube-rbac-proxy-self/0.log" Jul 07 00:31:09.292960 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.292930 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x5gks_4e40c58d-524f-4a93-9947-8296c8e84cbc/openshift-state-metrics/0.log" Jul 07 00:31:09.345081 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.345058 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/prometheus/0.log" Jul 07 00:31:09.374214 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.374156 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/config-reloader/0.log" Jul 07 00:31:09.399671 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.399651 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/thanos-sidecar/0.log" Jul 07 00:31:09.429028 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.428998 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/kube-rbac-proxy-web/0.log" Jul 07 00:31:09.456631 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.456607 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/kube-rbac-proxy/0.log" Jul 07 00:31:09.479519 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.479495 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/kube-rbac-proxy-thanos/0.log" Jul 07 00:31:09.501773 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.501694 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8594e540-5e09-43d5-8276-a810b5aa9e8b/init-config-reloader/0.log" Jul 07 00:31:09.721043 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.721017 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/thanos-query/0.log" Jul 07 00:31:09.741100 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.741078 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/kube-rbac-proxy-web/0.log" Jul 07 00:31:09.762374 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.762352 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/kube-rbac-proxy/0.log" Jul 07 00:31:09.784723 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.784703 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/prom-label-proxy/0.log" Jul 07 00:31:09.814794 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.814774 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/kube-rbac-proxy-rules/0.log" Jul 07 00:31:09.836038 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:09.836022 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8c99fb6f-dwxzb_86462d59-0a9d-4f88-aa49-7bbf331b5386/kube-rbac-proxy-metrics/0.log" Jul 07 00:31:11.119366 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:11.119339 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-ktkbj_3dd02b66-6506-48e3-b3d6-5738cbab656e/networking-console-plugin/0.log" Jul 07 00:31:11.756803 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:11.756774 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/1.log" Jul 07 00:31:11.762100 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:11.762079 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kjm87_cd94558a-fa48-47f1-b582-d14568f0d0a4/console-operator/2.log" Jul 07 00:31:12.448759 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.448716 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9"] Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449055 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8697f2e8-f8c7-463d-b86e-9fe05069fbcd" containerName="cleanup" Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449067 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8697f2e8-f8c7-463d-b86e-9fe05069fbcd" containerName="cleanup" Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449086 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58014652-e13e-401d-9597-bbefc798c82f" containerName="maas-api" Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449091 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58014652-e13e-401d-9597-bbefc798c82f" containerName="maas-api" Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449106 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7280b3c2-0395-438a-b81f-6c8064fe4b17" containerName="cleanup" Jul 07 00:31:12.449126 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449111 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7280b3c2-0395-438a-b81f-6c8064fe4b17" containerName="cleanup" Jul 07 00:31:12.449322 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449162 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7280b3c2-0395-438a-b81f-6c8064fe4b17" containerName="cleanup" Jul 07 00:31:12.449322 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449171 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8697f2e8-f8c7-463d-b86e-9fe05069fbcd" containerName="cleanup" Jul 07 00:31:12.449322 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.449178 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58014652-e13e-401d-9597-bbefc798c82f" containerName="maas-api" Jul 07 00:31:12.452243 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.452225 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.455028 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.455011 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrxzw\"/\"kube-root-ca.crt\"" Jul 07 00:31:12.456200 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.456182 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrxzw\"/\"openshift-service-ca.crt\"" Jul 07 00:31:12.456298 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.456216 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nrxzw\"/\"default-dockercfg-68rc2\"" Jul 07 00:31:12.461643 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.461617 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9"] Jul 07 00:31:12.516890 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.516841 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-podres\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.517059 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.516919 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-lib-modules\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.517059 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.516936 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-proc\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.517059 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.516953 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcls2\" (UniqueName: \"kubernetes.io/projected/5ea4c32b-b275-4699-821e-5e907db2c533-kube-api-access-hcls2\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.517059 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.516976 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-sys\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-lib-modules\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618149 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618150 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-proc\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618330 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcls2\" (UniqueName: \"kubernetes.io/projected/5ea4c32b-b275-4699-821e-5e907db2c533-kube-api-access-hcls2\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618330 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618194 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-sys\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618330 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618250 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-proc\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618330 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618269 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-podres\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618330 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618305 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-lib-modules\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618500 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618337 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-sys\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.618500 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.618358 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ea4c32b-b275-4699-821e-5e907db2c533-podres\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.626441 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.626413 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcls2\" (UniqueName: \"kubernetes.io/projected/5ea4c32b-b275-4699-821e-5e907db2c533-kube-api-access-hcls2\") pod \"perf-node-gather-daemonset-dk4d9\" (UID: \"5ea4c32b-b275-4699-821e-5e907db2c533\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.762223 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.762199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:12.889487 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:12.889406 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9"] Jul 07 00:31:12.892401 ip-10-0-132-169 kubenswrapper[2556]: W0707 00:31:12.892372 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ea4c32b_b275_4699_821e_5e907db2c533.slice/crio-ce7dee25127daa9fd7d09a851b8c2040503b5a65cb958a44eb55f535273cd8dc WatchSource:0}: Error finding container ce7dee25127daa9fd7d09a851b8c2040503b5a65cb958a44eb55f535273cd8dc: Status 404 returned error can't find the container with id ce7dee25127daa9fd7d09a851b8c2040503b5a65cb958a44eb55f535273cd8dc Jul 07 00:31:13.036614 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:13.036560 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" event={"ID":"5ea4c32b-b275-4699-821e-5e907db2c533","Type":"ContainerStarted","Data":"59866471c5b8e04e1187153870683ee8cc7547d3653a35ff03c6a4b64f455723"} Jul 07 00:31:13.036614 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:13.036605 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" event={"ID":"5ea4c32b-b275-4699-821e-5e907db2c533","Type":"ContainerStarted","Data":"ce7dee25127daa9fd7d09a851b8c2040503b5a65cb958a44eb55f535273cd8dc"} Jul 07 00:31:13.036794 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:13.036688 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:13.054439 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:13.054395 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" podStartSLOduration=1.054381679 podStartE2EDuration="1.054381679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:31:13.053056528 +0000 UTC m=+2539.983114031" watchObservedRunningTime="2026-07-07 00:31:13.054381679 +0000 UTC m=+2539.984439183" Jul 07 00:31:13.107320 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:13.107297 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-77dm4_983d417b-c423-4a1f-b287-bfc925dcf1c0/volume-data-source-validator/0.log" Jul 07 00:31:14.164512 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:14.164484 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mn5vl_73de84aa-74d2-4217-9e31-d069995b0a3a/dns/0.log" Jul 07 00:31:14.184961 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:14.184940 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mn5vl_73de84aa-74d2-4217-9e31-d069995b0a3a/kube-rbac-proxy/0.log" Jul 07 00:31:14.322958 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:14.322931 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpbff_47916393-3c42-4fff-ac7e-3c1a27df4df1/dns-node-resolver/0.log" Jul 07 00:31:15.002396 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:15.002363 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c847bf6bd-k62x9_7114e426-2c4b-4bbf-99f1-3ae91eed8f07/registry/0.log" Jul 07 00:31:15.050469 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:15.050446 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6nbc_1b4e1901-cdb3-4bc4-9c18-4acb64f2aa38/node-ca/0.log" Jul 07 00:31:16.095385 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:16.095355 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-99ckj_462f541c-54a8-41d2-abf1-fbcb1df4e80c/discovery/0.log" Jul 07 00:31:16.330001 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:16.329977 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9b6988657-sb8wz_c1bfd36e-da1a-49d8-859e-1c66d4bc3e98/router/0.log" Jul 07 00:31:17.054864 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.054829 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wmgj8_2392b3d1-2fbc-4704-9f40-60f4d730adf7/serve-healthcheck-canary/0.log" Jul 07 00:31:17.531116 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.531089 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-j68w9_0dca4762-89a1-4978-aef9-09093526d95e/insights-operator/0.log" Jul 07 00:31:17.532033 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.532014 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-j68w9_0dca4762-89a1-4978-aef9-09093526d95e/insights-operator/1.log" Jul 07 00:31:17.698184 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.698151 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wlrkh_bab54804-6342-4511-bc8a-281eba5cd430/kube-rbac-proxy/0.log" Jul 07 00:31:17.718417 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.718395 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wlrkh_bab54804-6342-4511-bc8a-281eba5cd430/exporter/0.log" Jul 07 00:31:17.742972 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:17.741460 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wlrkh_bab54804-6342-4511-bc8a-281eba5cd430/extractor/0.log" Jul 07 00:31:19.052184 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:19.052160 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-dk4d9" Jul 07 00:31:20.258343 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:20.258311 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-rbf8n_30826312-2250-4686-9100-868b4b1ff9c3/cleanup/0.log" Jul 07 00:31:20.291237 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:20.291211 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723070-nlx56_8697f2e8-f8c7-463d-b86e-9fe05069fbcd/cleanup/0.log" Jul 07 00:31:20.562054 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:20.561967 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-55bbr_b095a823-55df-482e-81a6-6ba7315876da/manager/0.log" Jul 07 00:31:20.585470 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:20.585449 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-55bbr_b095a823-55df-482e-81a6-6ba7315876da/copy-manifests/0.log" Jul 07 00:31:22.306406 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:22.306377 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9d77cc8c4-z4lbf_f6673faf-b9c5-4e88-a28f-6717b3dbd060/manager/0.log" Jul 07 00:31:22.748597 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:22.748563 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-99ckj_462f541c-54a8-41d2-abf1-fbcb1df4e80c/discovery/0.log" Jul 07 00:31:22.917243 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:22.917169 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9b6988657-sb8wz_c1bfd36e-da1a-49d8-859e-1c66d4bc3e98/router/0.log" Jul 07 00:31:24.167957 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:24.167915 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-99ckj_462f541c-54a8-41d2-abf1-fbcb1df4e80c/discovery/0.log" Jul 07 00:31:24.349672 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:24.349644 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9b6988657-sb8wz_c1bfd36e-da1a-49d8-859e-1c66d4bc3e98/router/0.log" Jul 07 00:31:27.631117 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:27.631078 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-w5vzz_b60ae206-9e76-4030-8791-5e6975a48b7d/kube-storage-version-migrator-operator/1.log" Jul 07 00:31:27.631741 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:27.631721 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-w5vzz_b60ae206-9e76-4030-8791-5e6975a48b7d/kube-storage-version-migrator-operator/0.log" Jul 07 00:31:28.762637 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.762603 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/kube-multus-additional-cni-plugins/0.log" Jul 07 00:31:28.782572 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.782547 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/egress-router-binary-copy/0.log" Jul 07 00:31:28.802680 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.802660 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/cni-plugins/0.log" Jul 07 00:31:28.822715 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.822690 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/bond-cni-plugin/0.log" Jul 07 00:31:28.843452 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.843429 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/routeoverride-cni/0.log" Jul 07 00:31:28.863654 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.863634 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/whereabouts-cni-bincopy/0.log" Jul 07 00:31:28.884787 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:28.884775 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cj74h_5415644a-9ad7-4222-9893-beed7eeb75a7/whereabouts-cni/0.log" Jul 07 00:31:29.144407 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:29.144334 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vhl7b_5fb4b4e8-8c8b-4a4c-8d35-cb4c8fd2ef24/kube-multus/0.log" Jul 07 00:31:29.173283 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:29.173255 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7xjzp_21436535-c81e-4688-bc63-5c0720cba1cd/network-metrics-daemon/0.log" Jul 07 00:31:29.192007 ip-10-0-132-169 kubenswrapper[2556]: I0707 00:31:29.191984 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7xjzp_21436535-c81e-4688-bc63-5c0720cba1cd/kube-rbac-proxy/0.log"