Jul 09 17:25:59.569952 ip-10-0-130-12 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 17:25:59.569968 ip-10-0-130-12 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 17:25:59.569979 ip-10-0-130-12 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 17:25:59.570313 ip-10-0-130-12 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 17:26:08.779517 ip-10-0-130-12 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 09 17:26:08.779665 ip-10-0-130-12 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d9ff545022fa47d4b2891cd7b6e46053 -- Jul 09 17:27:37.407673 ip-10-0-130-12 systemd[1]: Starting Kubernetes Kubelet... Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.869556 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873655 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873668 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873672 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873675 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873678 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873686 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873689 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.050918 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873692 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873694 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873697 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873700 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873702 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873705 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873708 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873710 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873714 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873718 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873721 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873724 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873727 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873730 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873733 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873736 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873739 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873741 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873744 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.219409 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873746 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:37.949528 ip-10-0-130-12 systemd[1]: Started Kubernetes Kubelet. Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873749 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873751 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873754 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873757 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873759 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873761 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873764 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873766 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873768 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873771 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873773 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873782 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873785 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873788 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873790 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873793 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873796 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873799 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873801 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.220754 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873804 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873806 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873809 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873812 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873814 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873818 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873821 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873823 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873826 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873829 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873831 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873834 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873836 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873839 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873842 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873844 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873847 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873849 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873852 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873855 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.221786 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873857 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873860 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873862 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873865 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873868 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873871 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873874 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873876 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873879 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873882 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873884 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873887 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873890 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873893 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873896 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873900 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873903 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873906 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873909 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.222799 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873912 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873915 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873918 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873920 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873923 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873925 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873928 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873930 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873933 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873935 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873938 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873940 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873943 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.873945 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874638 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874644 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874648 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874652 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874655 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874658 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.223519 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874660 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874663 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874666 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874668 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874671 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874673 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874677 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874680 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874682 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874685 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874689 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874693 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874696 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874699 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874702 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874705 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874707 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874710 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874713 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.224239 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874716 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874718 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874722 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874724 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874727 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874731 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874734 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874737 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874742 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874745 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874748 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874751 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874754 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874758 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874760 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874764 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874766 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874769 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874772 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.224859 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874775 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874777 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874780 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874782 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874785 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874787 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874790 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874792 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874795 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874797 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874799 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874802 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874804 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874807 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874809 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874812 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874814 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874817 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874819 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874821 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.225488 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874824 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874827 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874829 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874832 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874835 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874838 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874840 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874843 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874845 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874848 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874850 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874852 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874855 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874857 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874860 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874862 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874864 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874867 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874869 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874872 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.226110 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874874 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874877 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874880 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874882 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874884 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874887 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874890 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874892 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874896 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874899 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874902 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874904 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874907 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874910 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.874912 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875265 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875270 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875273 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875275 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875279 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.226796 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875281 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875283 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875286 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875289 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875291 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875294 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875298 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875302 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875305 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875308 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875311 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875313 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875316 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875318 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875321 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875323 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875326 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875328 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875331 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.227436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875333 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875336 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875342 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875345 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875348 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875351 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875354 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875358 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875360 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875380 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875383 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875385 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875388 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875391 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875394 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875396 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875399 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875401 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875404 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.228029 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875408 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875411 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875414 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875416 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875419 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875421 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875424 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875426 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875429 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875431 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875434 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875436 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875439 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875441 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875444 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875448 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875450 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875453 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875456 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875459 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.228624 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875461 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875464 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875467 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875470 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875473 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875475 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875478 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875481 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875483 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875486 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875489 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875492 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875494 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875497 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875499 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875501 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875504 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875506 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875509 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.229448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875512 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875514 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875517 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875519 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875522 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875524 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875527 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875529 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875533 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875536 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875538 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875541 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875543 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875546 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875548 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875551 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.875553 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875619 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875626 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875640 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875644 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875649 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 17:27:38.230509 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875652 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875656 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875660 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875663 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875666 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875669 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875674 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875677 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875680 2577 flags.go:64] FLAG: --cgroup-root="" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875683 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875685 2577 flags.go:64] FLAG: --client-ca-file="" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875688 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875691 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875695 2577 flags.go:64] FLAG: --cluster-domain="" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875698 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875701 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875704 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875707 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875713 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875716 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875719 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875722 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875725 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875728 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 17:27:38.231068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875731 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875734 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875737 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875741 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875744 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875746 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875749 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875752 2577 flags.go:64] FLAG: --enable-server="true" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875756 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875760 2577 flags.go:64] FLAG: --event-burst="100" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875763 2577 flags.go:64] FLAG: --event-qps="50" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875766 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875769 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875772 2577 flags.go:64] FLAG: --eviction-hard="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875777 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875780 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875783 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875786 2577 flags.go:64] FLAG: --eviction-soft="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875789 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875792 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875794 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875797 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875800 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875803 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875806 2577 flags.go:64] FLAG: --feature-gates="" Jul 09 17:27:38.231756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875809 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875814 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875817 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875820 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875822 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875825 2577 flags.go:64] FLAG: --help="false" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875828 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875831 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875834 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875837 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875840 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875843 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875846 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875848 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875851 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875854 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875857 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875860 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875863 2577 flags.go:64] FLAG: --kube-reserved="" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875866 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875869 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875873 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875876 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875878 2577 flags.go:64] FLAG: --lock-file="" Jul 09 17:27:38.232535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875881 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875884 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875886 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875892 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875894 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875897 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875900 2577 flags.go:64] FLAG: --logging-format="text" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875903 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875906 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875910 2577 flags.go:64] FLAG: --manifest-url="" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875913 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875917 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875920 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875924 2577 flags.go:64] FLAG: --max-pods="110" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875927 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875929 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875932 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875935 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875938 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875941 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875944 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875951 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875954 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875957 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 17:27:38.234756 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875960 2577 flags.go:64] FLAG: --pod-cidr="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875963 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875968 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875971 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875974 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875978 2577 flags.go:64] FLAG: --port="10250" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875981 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875984 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09a2e59747bd7eb8a" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875988 2577 flags.go:64] FLAG: --qos-reserved="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875991 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875994 2577 flags.go:64] FLAG: --register-node="true" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.875997 2577 flags.go:64] FLAG: --register-with-taints="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876000 2577 flags.go:64] FLAG: --registry-burst="10" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876003 2577 flags.go:64] FLAG: --registry-qps="5" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876006 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876009 2577 flags.go:64] FLAG: --reserved-memory="" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876012 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876015 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876019 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876022 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876024 2577 flags.go:64] FLAG: --runonce="false" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876027 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876030 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876033 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876036 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 17:27:38.243607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876039 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876042 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876045 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876048 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876051 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876054 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876057 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876059 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876062 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876066 2577 flags.go:64] FLAG: --system-cgroups="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876069 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876074 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876078 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876080 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876084 2577 flags.go:64] FLAG: --tls-min-version="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876087 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876089 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876092 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876095 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876097 2577 flags.go:64] FLAG: --v="2" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876101 2577 flags.go:64] FLAG: --version="false" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876105 2577 flags.go:64] FLAG: --vmodule="" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876109 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.876112 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876249 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.244616 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876255 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876258 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876261 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876263 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876267 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876270 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876273 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876276 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876278 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876280 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876283 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876287 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876290 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876293 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876296 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876299 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876302 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876311 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876314 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876318 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.397526 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876320 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876323 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876325 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876327 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876330 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876332 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876335 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876338 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876340 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876343 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876345 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876348 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876352 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876354 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876357 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876359 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876377 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876380 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876383 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.398210 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876385 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876388 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876390 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876392 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876395 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876398 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876400 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876403 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876405 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876408 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876410 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876418 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876422 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876425 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876427 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876430 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876432 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876435 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876437 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876440 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.398825 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876442 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876445 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876447 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876450 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876453 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876457 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876460 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876462 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876464 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876467 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876469 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876472 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876474 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876477 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876479 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876481 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876484 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876486 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876489 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876491 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.399448 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876494 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876496 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876499 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876501 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876513 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876516 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876518 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876521 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876523 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876526 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876528 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876531 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876533 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876536 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876539 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876541 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876543 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876549 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.399950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.876552 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.877270 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.886634 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.886649 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.886663 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.886673 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886733 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886739 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886742 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886746 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886749 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886752 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886755 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886757 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886760 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.400460 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886763 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886765 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886769 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886773 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886776 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886779 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886784 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886787 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886791 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886794 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886797 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886800 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886803 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886807 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886811 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886814 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886817 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886819 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886822 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.400870 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886824 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886827 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886830 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886832 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886835 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886838 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886840 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886843 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886846 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886848 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886851 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886853 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886856 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886858 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886861 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886863 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886866 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886868 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886873 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.401447 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886878 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886881 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886884 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886887 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886889 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886892 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886895 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886898 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886902 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886905 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886907 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886910 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886912 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886915 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886917 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886920 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886922 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886925 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886928 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886930 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.401948 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886932 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886935 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886937 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886940 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886943 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886945 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886947 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886950 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886952 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886955 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886957 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886960 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886964 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886967 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886969 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886972 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886974 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886977 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886980 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.402527 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886982 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886986 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886988 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886991 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886993 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886996 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.886998 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887001 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887003 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887005 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887008 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887010 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887013 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.887018 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887167 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:27:38.403011 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887173 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887176 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887179 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887182 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887185 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887187 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887190 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887193 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887195 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887198 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887200 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887206 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887208 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887212 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887214 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887217 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887220 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887223 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887226 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887229 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:27:38.403461 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887232 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887234 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887237 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887240 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887242 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887245 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887247 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887250 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887253 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887256 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887259 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887261 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887263 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887266 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887268 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887271 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887273 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887276 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887278 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887281 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:27:38.403960 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887283 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887287 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887290 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887294 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887297 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887300 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887303 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887306 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887308 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887311 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887315 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887317 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887320 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887322 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887325 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887327 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887330 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887332 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887334 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:27:38.404499 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887337 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887339 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887342 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887344 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887347 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887349 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887351 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887354 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887357 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887359 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887380 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887384 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887387 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887390 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887392 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887395 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887398 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887401 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887406 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:27:38.405042 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887409 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887412 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887415 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887418 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887422 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887425 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887428 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887430 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887432 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887435 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887437 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887440 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887442 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887445 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887447 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887450 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887452 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887455 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887457 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:27:38.405584 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:37.887460 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.887464 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.887579 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.890101 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.891821 2577 server.go:1019] "Starting client certificate rotation" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.891932 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.891998 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.894698 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.895836 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.908476 2577 log.go:25] "Validated CRI v1 runtime API" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.911940 2577 log.go:25] "Validated CRI v1 image API" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.913141 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 17:27:38.406073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.917520 2577 fs.go:135] Filesystem UUIDs: map[6b541531-0862-43fc-8e49-73d389cc437a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f03722b6-2301-4d88-b23c-25897ee65ce8:/dev/nvme0n1p3] Jul 09 17:27:38.406493 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.917537 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 17:27:38.406493 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.920692 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.924027 2577 manager.go:216] Machine: {Timestamp:2026-07-09 17:27:37.922634356 +0000 UTC m=+0.371686750 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200466 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2929ce99defac9245e4ed44efd127b SystemUUID:ec2929ce-99de-fac9-245e-4ed44efd127b BootID:d9ff5450-22fa-47d4-b289-1cd7b6e46053 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:a8:e5:e3:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:a8:e5:e3:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:2b:b0:68:40:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.924124 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.924199 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.927792 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.927817 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.928716 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.928725 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.928736 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.929568 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.931409 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936088 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936211 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936238 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936248 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936266 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936276 2577 kubelet.go:398] "Adding apiserver pod source" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936285 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 17:27:38.406645 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936645 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.936659 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.939114 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.942862 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.942902 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943109 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943123 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943128 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943134 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943142 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943148 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943707 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943716 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943726 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943734 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943750 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.943761 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.944729 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.944738 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.948603 2577 server.go:1268] "Started kubelet" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.948668 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.948708 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.948912 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.948950 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.949597 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.949597 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.950262 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.955457 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.958436 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.958162 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-12.ec2.internal.18c0af2c76f9d368 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-12.ec2.internal,UID:ip-10-0-130-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-12.ec2.internal,},FirstTimestamp:2026-07-09 17:27:37.948222312 +0000 UTC m=+0.397274704,LastTimestamp:2026-07-09 17:27:37.948222312 +0000 UTC m=+0.397274704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-12.ec2.internal,}" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.960128 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.960144 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.960240 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961013 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961032 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961013 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961138 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961148 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961141 2577 factory.go:55] Registering systemd factory Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961171 2577 factory.go:223] Registration of the systemd container factory successfully Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961352 2577 factory.go:153] Registering CRI-O factory Jul 09 17:27:38.407419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961377 2577 factory.go:223] Registration of the crio container factory successfully Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961422 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961441 2577 factory.go:103] Registering Raw factory Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961464 2577 manager.go:1196] Started watching for new ooms in manager Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.961857 2577 manager.go:349] Starting recovery of all containers Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.963837 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.963880 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.965444 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqh7t" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.971035 2577 manager.go:354] Recovery completed Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.972424 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqh7t" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.976075 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.977970 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978004 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978019 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978533 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978540 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978542 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.978611 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.979223 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.979248 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.979257 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:37.979668 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-12.ec2.internal.18c0af2c78bffaf7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-12.ec2.internal,UID:ip-10-0-130-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-12.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-12.ec2.internal,},FirstTimestamp:2026-07-09 17:27:37.977985783 +0000 UTC m=+0.427038182,LastTimestamp:2026-07-09 17:27:37.977985783 +0000 UTC m=+0.427038182,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-12.ec2.internal,}" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.980616 2577 policy_none.go:49] "None policy: Start" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.980629 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.980639 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:37.982909 2577 policy_none.go:47] "Start" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.061325 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.062914 2577 manager.go:337] "Starting Device Plugin manager" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.062942 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.062951 2577 server.go:87] "Starting device plugin registration server" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.063224 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.063236 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.063340 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.063427 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.063434 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.063898 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.063953 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.066708 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.067860 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 17:27:38.408489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.067893 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.067915 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.067955 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.070208 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.163733 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.164718 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.164747 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.164756 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.164779 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.169024 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal"] Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.169082 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.169872 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.169895 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.169906 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.171871 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-12.ec2.internal\" not found" node="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.172171 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.172288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.172319 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173019 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173026 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173045 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173048 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173059 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.173072 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.176203 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.176218 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-12.ec2.internal\": node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.189053 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-12.ec2.internal\" not found" node="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.195341 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.218728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.218759 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.219570 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.219596 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.219610 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.231309 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-12.ec2.internal\" not found" node="ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.261641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.528696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.261666 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.261682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df20e6c48b1fb1cb9b5d2a27957e95a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-12.ec2.internal\" (UID: \"df20e6c48b1fb1cb9b5d2a27957e95a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.296390 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df20e6c48b1fb1cb9b5d2a27957e95a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-12.ec2.internal\" (UID: \"df20e6c48b1fb1cb9b5d2a27957e95a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362899 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df20e6c48b1fb1cb9b5d2a27957e95a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-12.ec2.internal\" (UID: \"df20e6c48b1fb1cb9b5d2a27957e95a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.362907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a2566d6b667c14d54da2b5aaa448787-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal\" (UID: \"3a2566d6b667c14d54da2b5aaa448787\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.397058 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.497679 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.530631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.518850 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.534236 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.533967 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.631858 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.598741 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.752625 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.699283 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-12.ec2.internal\" not found" Jul 09 17:27:38.883319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.769838 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:38.883319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.860675 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.883319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.869238 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:27:38.883319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.870665 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" Jul 09 17:27:38.883319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.882250 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.891428 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.891547 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.891580 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.937056 2577 apiserver.go:52] "Watching apiserver" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.953488 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.954442 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-sjr8l","openshift-image-registry/node-ca-kjvx5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal","openshift-multus/multus-additional-cni-plugins-bfldg","openshift-ovn-kubernetes/ovnkube-node-5q49f","kube-system/global-pull-secret-syncer-2dbb5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg","openshift-multus/multus-n9s5d","openshift-multus/network-metrics-daemon-t5g2n","openshift-network-diagnostics/network-check-target-bwx97","openshift-network-operator/iptables-alerter-8n965","kube-system/konnectivity-agent-csfxj","kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal","openshift-cluster-node-tuning-operator/tuned-4cv45"] Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.956800 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.962897 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.963262 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.963356 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.963391 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.963403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tfntx\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.966466 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.966903 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.967020 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.967068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.967196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8wjsp\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83651f0-ec52-46a6-9671-24fef94d000b-tmp-dir\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968161 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-host\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968176 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-serviceca\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzxgj\" (UniqueName: \"kubernetes.io/projected/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-kube-api-access-nzxgj\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b83651f0-ec52-46a6-9671-24fef94d000b-hosts-file\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968258 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9mh7\" (UniqueName: \"kubernetes.io/projected/b83651f0-ec52-46a6-9671-24fef94d000b-kube-api-access-m9mh7\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968271 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968497 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968674 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 17:27:39.023706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968708 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.968909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.969213 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qcpc4\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.969400 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.971094 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.971133 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.971451 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.971576 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.971710 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h8bbk\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.971825 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.972021 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.972145 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.972257 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.973273 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.974262 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.974961 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.975379 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.975695 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.975818 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4hkht\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.975932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.977104 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:22:37 +0000 UTC" deadline="2028-04-16 03:00:02.0210583 +0000 UTC" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.977140 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15513h32m23.043920123s" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.977300 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.977520 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z8fcr\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.977530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.977574 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.979727 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:38.979785 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.981880 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.984084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.984145 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.984171 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 17:27:39.024832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.984172 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.984436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nft6m\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.986235 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.986433 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.986774 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.986871 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bx54m\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.988465 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.988802 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.988995 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9q7fj\"" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:38.999855 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9q7nt" Jul 09 17:27:39.025875 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.005656 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9q7nt" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.067033 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.068901 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.068926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-socket-dir-parent\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.068946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-hostroot\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.068968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-etc-selinux\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.068988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-systemd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069007 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-bin\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx5jb\" (UniqueName: \"kubernetes.io/projected/047648cf-d30a-4556-aa48-5ba980836a26-kube-api-access-kx5jb\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-os-release\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-etc-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-node-log\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovn-node-metrics-cert\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-dbus\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-sys-fs\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b83651f0-ec52-46a6-9671-24fef94d000b-hosts-file\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.112492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610eec77-0088-4090-a67b-5208bebddadb-iptables-alerter-script\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7c79980-b4ef-4f51-9c50-526b6b290205-konnectivity-ca\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-multus\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-multus-daemon-config\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069305 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b83651f0-ec52-46a6-9671-24fef94d000b-hosts-file\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-device-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-kubelet-config\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-systemd\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-lib-modules\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069445 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-var-lib-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-log-socket\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-serviceca\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzxgj\" (UniqueName: \"kubernetes.io/projected/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-kube-api-access-nzxgj\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-env-overrides\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-k8s-cni-cncf-io\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-multus-certs\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113413 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069667 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-etc-kubernetes\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-sys\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-var-lib-kubelet\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6gxm\" (UniqueName: \"kubernetes.io/projected/26b40afe-2d93-4b89-b2c3-c7bc889e230d-kube-api-access-q6gxm\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-host\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv77w\" (UniqueName: \"kubernetes.io/projected/610eec77-0088-4090-a67b-5208bebddadb-kube-api-access-nv77w\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cnibin\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-host\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-config\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-system-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-netns\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-modprobe-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-ovn\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.113966 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.069986 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-script-lib\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g7hj\" (UniqueName: \"kubernetes.io/projected/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kube-api-access-4g7hj\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83651f0-ec52-46a6-9671-24fef94d000b-tmp-dir\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610eec77-0088-4090-a67b-5208bebddadb-host-slash\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-kubernetes\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-tmp\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsdsq\" (UniqueName: \"kubernetes.io/projected/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-kube-api-access-xsdsq\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-kubelet\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9mh7\" (UniqueName: \"kubernetes.io/projected/b83651f0-ec52-46a6-9671-24fef94d000b-kube-api-access-m9mh7\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-host\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-netd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070414 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-cnibin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070430 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-os-release\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070467 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.114585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7c79980-b4ef-4f51-9c50-526b6b290205-agent-certs\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-conf\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-run\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070549 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-systemd-units\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fqnx\" (UniqueName: \"kubernetes.io/projected/a6b88fc1-61bd-40c3-8a51-0d4153c69782-kube-api-access-6fqnx\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-socket-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysconfig\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070632 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpqcf\" (UniqueName: \"kubernetes.io/projected/431e5c77-b3ad-49b7-87d9-95ee5648429b-kube-api-access-hpqcf\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-tuned\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070689 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-system-cni-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-cni-binary-copy\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070732 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-bin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-kubelet\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.115150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-conf-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-registration-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-slash\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.070846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-netns\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.072441 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-serviceca\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.073215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83651f0-ec52-46a6-9671-24fef94d000b-tmp-dir\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.089321 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.092361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzxgj\" (UniqueName: \"kubernetes.io/projected/7d19f4e2-714d-458f-8d05-e5b2b92b03ab-kube-api-access-nzxgj\") pod \"node-ca-kjvx5\" (UID: \"7d19f4e2-714d-458f-8d05-e5b2b92b03ab\") " pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.092411 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9mh7\" (UniqueName: \"kubernetes.io/projected/b83651f0-ec52-46a6-9671-24fef94d000b-kube-api-access-m9mh7\") pod \"node-resolver-sjr8l\" (UID: \"b83651f0-ec52-46a6-9671-24fef94d000b\") " pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.114857 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-env-overrides\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-k8s-cni-cncf-io\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-multus-certs\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-etc-kubernetes\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-sys\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-k8s-cni-cncf-io\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-multus-certs\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.191209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-var-lib-kubelet\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-var-lib-kubelet\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-etc-kubernetes\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6gxm\" (UniqueName: \"kubernetes.io/projected/26b40afe-2d93-4b89-b2c3-c7bc889e230d-kube-api-access-q6gxm\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv77w\" (UniqueName: \"kubernetes.io/projected/610eec77-0088-4090-a67b-5208bebddadb-kube-api-access-nv77w\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cnibin\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-config\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-system-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171589 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-env-overrides\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-netns\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-modprobe-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cnibin\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171705 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-sys\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171756 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-system-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.171981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-ovn\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.192836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-script-lib\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g7hj\" (UniqueName: \"kubernetes.io/projected/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kube-api-access-4g7hj\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610eec77-0088-4090-a67b-5208bebddadb-host-slash\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-kubernetes\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-tmp\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172130 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsdsq\" (UniqueName: \"kubernetes.io/projected/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-kube-api-access-xsdsq\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-kubelet\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-host\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-run-netns\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-netd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-cnibin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-os-release\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.194081 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7c79980-b4ef-4f51-9c50-526b6b290205-agent-certs\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-conf\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172279 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-modprobe-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172291 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-run\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172320 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-kubelet\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-systemd-units\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-run\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172368 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fqnx\" (UniqueName: \"kubernetes.io/projected/a6b88fc1-61bd-40c3-8a51-0d4153c69782-kube-api-access-6fqnx\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172145 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-binary-copy\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-socket-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172427 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-host\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysconfig\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172463 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysconfig\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpqcf\" (UniqueName: \"kubernetes.io/projected/431e5c77-b3ad-49b7-87d9-95ee5648429b-kube-api-access-hpqcf\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172531 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-cni-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.194725 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-tuned\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172562 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-system-cni-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172556 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-netd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-cni-binary-copy\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-cnibin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-config\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-os-release\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172672 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-ovn\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172711 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-bin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-bin\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-kubelet\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-conf-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172811 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-registration-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-slash\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-netns\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.195787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-socket-dir-parent\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172916 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-hostroot\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-etc-selinux\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-systemd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.172986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-bin\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-conf\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx5jb\" (UniqueName: \"kubernetes.io/projected/047648cf-d30a-4556-aa48-5ba980836a26-kube-api-access-kx5jb\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173076 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovnkube-script-lib\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-os-release\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-etc-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-node-log\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovn-node-metrics-cert\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-dbus\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-sys-fs\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173204 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-kubelet\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-conf-dir\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/610eec77-0088-4090-a67b-5208bebddadb-host-slash\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173322 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-multus-socket-dir-parent\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173360 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-kubernetes\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-hostroot\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-sysctl-d\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173469 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-etc-selinux\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173495 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-run-systemd\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-cni-bin\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173552 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-etc-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610eec77-0088-4090-a67b-5208bebddadb-iptables-alerter-script\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.173617 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7c79980-b4ef-4f51-9c50-526b6b290205-konnectivity-ca\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-multus\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-multus-daemon-config\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.173681 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:27:39.673656497 +0000 UTC m=+2.122708876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:39.200905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-device-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-kubelet-config\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-systemd\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-lib-modules\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-var-lib-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173822 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-log-socket\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.173935 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.173978 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:27:39.673963119 +0000 UTC m=+2.123015503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.174015 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-multus-daemon-config\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.174027 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-os-release\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.174054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-node-log\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183038 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-device-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183058 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-kubelet-config\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-systemd\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/21fe8f0d-338f-4886-bed2-a7ec244552ba-dbus\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.201446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-sys-fs\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183444 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183453 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047648cf-d30a-4556-aa48-5ba980836a26-host-var-lib-cni-multus\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183476 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26b40afe-2d93-4b89-b2c3-c7bc889e230d-lib-modules\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183530 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-var-lib-openvswitch\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-socket-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183646 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7c79980-b4ef-4f51-9c50-526b6b290205-konnectivity-ca\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183717 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047648cf-d30a-4556-aa48-5ba980836a26-cni-binary-copy\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183717 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-system-cni-dir\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.183759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-slash\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.173284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-host-run-netns\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.184099 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.184130 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.184225 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.184227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/610eec77-0088-4090-a67b-5208bebddadb-iptables-alerter-script\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.184288 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:27:39.684271355 +0000 UTC m=+2.133323743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.184245 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.202065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.184654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-registration-dir\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.184683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-log-socket\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.184712 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6b88fc1-61bd-40c3-8a51-0d4153c69782-systemd-units\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.185271 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-tmp\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.185324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7c79980-b4ef-4f51-9c50-526b6b290205-agent-certs\") pod \"konnectivity-agent-csfxj\" (UID: \"d7c79980-b4ef-4f51-9c50-526b6b290205\") " pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.185604 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6b88fc1-61bd-40c3-8a51-0d4153c69782-ovn-node-metrics-cert\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.186673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26b40afe-2d93-4b89-b2c3-c7bc889e230d-etc-tuned\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.188826 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6gxm\" (UniqueName: \"kubernetes.io/projected/26b40afe-2d93-4b89-b2c3-c7bc889e230d-kube-api-access-q6gxm\") pod \"tuned-4cv45\" (UID: \"26b40afe-2d93-4b89-b2c3-c7bc889e230d\") " pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.189098 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx5jb\" (UniqueName: \"kubernetes.io/projected/047648cf-d30a-4556-aa48-5ba980836a26-kube-api-access-kx5jb\") pod \"multus-n9s5d\" (UID: \"047648cf-d30a-4556-aa48-5ba980836a26\") " pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.190285 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g7hj\" (UniqueName: \"kubernetes.io/projected/cdbfedda-d4b5-408b-9cfa-9a884b0a7a95-kube-api-access-4g7hj\") pod \"aws-ebs-csi-driver-node-xd2wg\" (UID: \"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.192546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsdsq\" (UniqueName: \"kubernetes.io/projected/3ad28ce5-02b2-49ad-ab0e-c6d1a914f299-kube-api-access-xsdsq\") pod \"multus-additional-cni-plugins-bfldg\" (UID: \"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299\") " pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.192849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fqnx\" (UniqueName: \"kubernetes.io/projected/a6b88fc1-61bd-40c3-8a51-0d4153c69782-kube-api-access-6fqnx\") pod \"ovnkube-node-5q49f\" (UID: \"a6b88fc1-61bd-40c3-8a51-0d4153c69782\") " pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.193559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpqcf\" (UniqueName: \"kubernetes.io/projected/431e5c77-b3ad-49b7-87d9-95ee5648429b-kube-api-access-hpqcf\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.202692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.201698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv77w\" (UniqueName: \"kubernetes.io/projected/610eec77-0088-4090-a67b-5208bebddadb-kube-api-access-nv77w\") pod \"iptables-alerter-8n965\" (UID: \"610eec77-0088-4090-a67b-5208bebddadb\") " pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.389137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.300733 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sjr8l" Jul 09 17:27:39.389137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.317705 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjvx5" Jul 09 17:27:39.389137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.338506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bfldg" Jul 09 17:27:39.389137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.357181 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:27:39.389137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.372961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.409751 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b88fc1_61bd_40c3_8a51_0d4153c69782.slice/crio-8093838672dadb13f941492ccd41235649ecce2b6e2df9de88734f25fa6dd38b WatchSource:0}: Error finding container 8093838672dadb13f941492ccd41235649ecce2b6e2df9de88734f25fa6dd38b: Status 404 returned error can't find the container with id 8093838672dadb13f941492ccd41235649ecce2b6e2df9de88734f25fa6dd38b Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.410231 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2566d6b667c14d54da2b5aaa448787.slice/crio-609cc3760dafefcf9610d197fa2b786175d66ea2d19b57fa2e8aea7e70be7032 WatchSource:0}: Error finding container 609cc3760dafefcf9610d197fa2b786175d66ea2d19b57fa2e8aea7e70be7032: Status 404 returned error can't find the container with id 609cc3760dafefcf9610d197fa2b786175d66ea2d19b57fa2e8aea7e70be7032 Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.414258 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.422015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9s5d" Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.431112 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047648cf_d30a_4556_aa48_5ba980836a26.slice/crio-4f551a7b234a340a12eb26feec2c93e3ee47b0aba2a5eb94d75cff8e9dabf4f6 WatchSource:0}: Error finding container 4f551a7b234a340a12eb26feec2c93e3ee47b0aba2a5eb94d75cff8e9dabf4f6: Status 404 returned error can't find the container with id 4f551a7b234a340a12eb26feec2c93e3ee47b0aba2a5eb94d75cff8e9dabf4f6 Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.445876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n965" Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.458902 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610eec77_0088_4090_a67b_5208bebddadb.slice/crio-c2df8c27beb0d688aa92d017c1a944aff8d3ba6d044f12fca6530be4595faf90 WatchSource:0}: Error finding container c2df8c27beb0d688aa92d017c1a944aff8d3ba6d044f12fca6530be4595faf90: Status 404 returned error can't find the container with id c2df8c27beb0d688aa92d017c1a944aff8d3ba6d044f12fca6530be4595faf90 Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.462043 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.464722 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4cv45" Jul 09 17:27:39.507220 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.469734 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c79980_b4ef_4f51_9c50_526b6b290205.slice/crio-a84dae60abd189c4b33bfa6331a025a7a12a7e8c6819b025bb6b73dd58c3e318 WatchSource:0}: Error finding container a84dae60abd189c4b33bfa6331a025a7a12a7e8c6819b025bb6b73dd58c3e318: Status 404 returned error can't find the container with id a84dae60abd189c4b33bfa6331a025a7a12a7e8c6819b025bb6b73dd58c3e318 Jul 09 17:27:39.592817 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.535857 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.677739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.677790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.677884 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.677887 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.677941 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:27:40.677928088 +0000 UTC m=+3.126980469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.678011 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:27:40.677989762 +0000 UTC m=+3.127042150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:39.741585 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.712882 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83651f0_ec52_46a6_9671_24fef94d000b.slice/crio-44916897b6aa82d722d8073523a348c955f81bf23174cd273916a159d4e3a186 WatchSource:0}: Error finding container 44916897b6aa82d722d8073523a348c955f81bf23174cd273916a159d4e3a186: Status 404 returned error can't find the container with id 44916897b6aa82d722d8073523a348c955f81bf23174cd273916a159d4e3a186 Jul 09 17:27:39.851652 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:39.779780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:39.851652 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.779907 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:39.851652 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.779922 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:39.851652 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.779931 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:39.851652 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:39.779979 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:27:40.779964689 +0000 UTC m=+3.229017270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:39.915755 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:27:39.908507 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbfedda_d4b5_408b_9cfa_9a884b0a7a95.slice/crio-ccc31f938476a701b8245a9588334b94b45aa5f2cb5979f7499d87a238ff60ac WatchSource:0}: Error finding container ccc31f938476a701b8245a9588334b94b45aa5f2cb5979f7499d87a238ff60ac: Status 404 returned error can't find the container with id ccc31f938476a701b8245a9588334b94b45aa5f2cb5979f7499d87a238ff60ac Jul 09 17:27:40.009041 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.008933 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:22:39 +0000 UTC" deadline="2028-01-13 16:36:47.584817661 +0000 UTC" Jul 09 17:27:40.009041 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.008988 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13271h9m7.575833675s" Jul 09 17:27:40.068546 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.068514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:40.068702 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.068520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:40.068702 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.068635 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:40.068789 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.068719 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:40.072860 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.072813 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjvx5" event={"ID":"7d19f4e2-714d-458f-8d05-e5b2b92b03ab","Type":"ContainerStarted","Data":"46abb5fccb20004a84df067cfcd75e4ae357a4abd1cf07fb5cd1c2c2037a0a80"} Jul 09 17:27:40.074573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.074545 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sjr8l" event={"ID":"b83651f0-ec52-46a6-9671-24fef94d000b","Type":"ContainerStarted","Data":"44916897b6aa82d722d8073523a348c955f81bf23174cd273916a159d4e3a186"} Jul 09 17:27:40.075666 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.075631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" event={"ID":"df20e6c48b1fb1cb9b5d2a27957e95a4","Type":"ContainerStarted","Data":"cb4406757e1e581ca6e84e1e4da74a1948601c0479664727c7985b2c97cf10fa"} Jul 09 17:27:40.077204 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.077145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4cv45" event={"ID":"26b40afe-2d93-4b89-b2c3-c7bc889e230d","Type":"ContainerStarted","Data":"cadc767b6c2c484d8ed367a59d61393160f6bff2ee85edae3cee983b3d18cd47"} Jul 09 17:27:40.079610 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.079570 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9s5d" event={"ID":"047648cf-d30a-4556-aa48-5ba980836a26","Type":"ContainerStarted","Data":"4f551a7b234a340a12eb26feec2c93e3ee47b0aba2a5eb94d75cff8e9dabf4f6"} Jul 09 17:27:40.080830 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.080798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" event={"ID":"3a2566d6b667c14d54da2b5aaa448787","Type":"ContainerStarted","Data":"609cc3760dafefcf9610d197fa2b786175d66ea2d19b57fa2e8aea7e70be7032"} Jul 09 17:27:40.082099 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.082009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"8093838672dadb13f941492ccd41235649ecce2b6e2df9de88734f25fa6dd38b"} Jul 09 17:27:40.083477 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.083411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" event={"ID":"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95","Type":"ContainerStarted","Data":"ccc31f938476a701b8245a9588334b94b45aa5f2cb5979f7499d87a238ff60ac"} Jul 09 17:27:40.084874 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.084849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-csfxj" event={"ID":"d7c79980-b4ef-4f51-9c50-526b6b290205","Type":"ContainerStarted","Data":"a84dae60abd189c4b33bfa6331a025a7a12a7e8c6819b025bb6b73dd58c3e318"} Jul 09 17:27:40.085922 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.085903 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n965" event={"ID":"610eec77-0088-4090-a67b-5208bebddadb","Type":"ContainerStarted","Data":"c2df8c27beb0d688aa92d017c1a944aff8d3ba6d044f12fca6530be4595faf90"} Jul 09 17:27:40.403507 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.403478 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:40.676675 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.676480 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.685139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.685185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.685290 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.685342 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:27:42.685324536 +0000 UTC m=+5.134376931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.685721 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:40.685806 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.685767 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:27:42.685752914 +0000 UTC m=+5.134805294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:40.785931 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:40.785887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:40.786108 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.786083 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:40.786108 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.786106 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:40.786226 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.786118 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:40.786226 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:40.786171 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:27:42.786153185 +0000 UTC m=+5.235205578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:41.009534 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:41.009439 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:22:39 +0000 UTC" deadline="2027-12-24 04:51:03.459903393 +0000 UTC" Jul 09 17:27:41.009534 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:41.009477 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12779h23m22.450430642s" Jul 09 17:27:41.069049 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:41.069017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:41.069216 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:41.069141 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:42.068495 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:42.068461 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:42.068495 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:42.068499 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:42.069019 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.068617 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:42.069080 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.069055 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:42.702416 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:42.702354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:42.702633 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:42.702441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:42.702633 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.702607 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:42.702746 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.702659 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:42.702746 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.702690 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:27:46.702670255 +0000 UTC m=+9.151722650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:42.702746 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.702712 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:27:46.702697509 +0000 UTC m=+9.151749893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:42.803491 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:42.802815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:42.803491 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.803020 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:42.803491 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.803039 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:42.803491 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.803052 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:42.803491 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:42.803116 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:27:46.803096997 +0000 UTC m=+9.252149434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:43.069132 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:43.068643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:43.069132 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:43.068766 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:44.069829 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:44.069798 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:44.070269 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:44.069946 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:44.070342 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:44.069803 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:44.070415 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:44.070383 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:45.068433 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:45.068396 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:45.068604 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:45.068542 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:46.069077 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:46.069042 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:46.069077 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:46.069060 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:46.069646 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.069184 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:46.069646 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.069299 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:46.731657 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:46.731586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:46.731657 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:46.731632 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:46.731874 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.731747 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:46.731874 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.731749 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:46.731874 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.731807 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:27:54.731789924 +0000 UTC m=+17.180842316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:46.731874 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.731824 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:27:54.731815507 +0000 UTC m=+17.180867888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:46.832631 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:46.832572 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:46.832814 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.832736 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:46.832814 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.832765 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:46.832814 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.832779 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:46.832971 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:46.832836 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:27:54.832817552 +0000 UTC m=+17.281869936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:47.068732 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:47.068649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:47.068885 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:47.068790 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:48.070896 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:48.070248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:48.070896 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:48.070360 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:48.070896 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:48.070739 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:48.070896 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:48.070841 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:49.068129 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:49.068095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:49.068327 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:49.068218 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:50.069195 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.068971 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:50.069664 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.068983 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:50.069664 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:50.069330 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:50.069664 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:50.069360 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:50.114847 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.114758 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" event={"ID":"df20e6c48b1fb1cb9b5d2a27957e95a4","Type":"ContainerStarted","Data":"188fe1778a530d7e7fad0de3b16a9c4cc35d82762d928cccdd3ccc919c87e052"} Jul 09 17:27:50.116735 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.116708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4cv45" event={"ID":"26b40afe-2d93-4b89-b2c3-c7bc889e230d","Type":"ContainerStarted","Data":"8975df68356c9d02ab4f1aaf68e460b346c7617634fc52e955416daed3d5c7c8"} Jul 09 17:27:50.117962 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.117939 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerStarted","Data":"aa16a4831b69386b02fd5322e0cd1eff6052f176a11b1e023db8200ae9c34f35"} Jul 09 17:27:50.143006 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.142954 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-12.ec2.internal" podStartSLOduration=12.142938067 podStartE2EDuration="12.142938067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:50.12864988 +0000 UTC m=+12.577702282" watchObservedRunningTime="2026-07-09 17:27:50.142938067 +0000 UTC m=+12.591990470" Jul 09 17:27:50.143177 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:50.143049 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4cv45" podStartSLOduration=1.949479985 podStartE2EDuration="12.143039458s" totalImagesPullingTime="10.193559473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:50.142923421 +0000 UTC m=+12.591975876" watchObservedRunningTime="2026-07-09 17:27:50.143039458 +0000 UTC m=+12.592091860" Jul 09 17:27:51.068466 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.068424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:51.068693 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:51.068561 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:51.123537 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.123209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" event={"ID":"3a2566d6b667c14d54da2b5aaa448787","Type":"ContainerStarted","Data":"d92838ef4b5a2d1e594b2e787b08b77054d5e4b82e63f09c5377ef549388f3a9"} Jul 09 17:27:51.125666 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.125638 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" event={"ID":"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95","Type":"ContainerStarted","Data":"18ca6b2c40f47f1fe4941b831ecbd2dbf063593cc57b2102b89e3023a4fcc419"} Jul 09 17:27:51.127730 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.127703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-csfxj" event={"ID":"d7c79980-b4ef-4f51-9c50-526b6b290205","Type":"ContainerStarted","Data":"dec5c01625bdc51105b6ee46001b2b4a6535d7230574eb7ed52a286bced78a6e"} Jul 09 17:27:51.130390 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.130353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjvx5" event={"ID":"7d19f4e2-714d-458f-8d05-e5b2b92b03ab","Type":"ContainerStarted","Data":"1ea946327f19164cf317f3006b203c81f6002421bc5b60df607731a811f15b45"} Jul 09 17:27:51.132463 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.132439 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sjr8l" event={"ID":"b83651f0-ec52-46a6-9671-24fef94d000b","Type":"ContainerStarted","Data":"5783f1ef866d7e6bd712e8e2a2c1fca5900a8da5b94af98ccf960a89ef22c0d1"} Jul 09 17:27:51.162907 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.162850 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sjr8l" podStartSLOduration=3.207422266 podStartE2EDuration="13.162831014s" totalImagesPullingTime="9.955408748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:51.149582244 +0000 UTC m=+13.598634647" watchObservedRunningTime="2026-07-09 17:27:51.162831014 +0000 UTC m=+13.611883417" Jul 09 17:27:51.163063 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.162958 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kjvx5" podStartSLOduration=3.362083671 podStartE2EDuration="13.162947593s" totalImagesPullingTime="9.800863922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:51.162239206 +0000 UTC m=+13.611291627" watchObservedRunningTime="2026-07-09 17:27:51.162947593 +0000 UTC m=+13.611999995" Jul 09 17:27:51.174873 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:51.174819 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-csfxj" podStartSLOduration=3.002182098 podStartE2EDuration="13.174801578s" totalImagesPullingTime="10.17261948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:51.174679198 +0000 UTC m=+13.623731627" watchObservedRunningTime="2026-07-09 17:27:51.174801578 +0000 UTC m=+13.623854031" Jul 09 17:27:52.068827 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:52.068794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:52.069021 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:52.068799 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:52.069021 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:52.068924 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:52.069138 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:52.069013 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:52.136585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:52.136488 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n965" event={"ID":"610eec77-0088-4090-a67b-5208bebddadb","Type":"ContainerStarted","Data":"9cd50aac3796689a558852c80a18181f2848a0c5d341d7ae5152c3fa74483ac2"} Jul 09 17:27:52.152479 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:52.152426 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8n965" podStartSLOduration=3.936708661 podStartE2EDuration="14.152398014s" totalImagesPullingTime="10.215689353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:27:52.151575796 +0000 UTC m=+14.600628198" watchObservedRunningTime="2026-07-09 17:27:52.152398014 +0000 UTC m=+14.601450431" Jul 09 17:27:53.069052 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:53.069018 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:53.069242 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:53.069152 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:54.068570 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:54.068350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:54.069033 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:54.068442 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:54.069033 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.068670 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:54.069033 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.068779 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:54.799651 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:54.799561 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:54.799651 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:54.799630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:54.799965 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.799740 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:54.799965 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.799770 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:54.799965 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.799821 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:28:10.799799022 +0000 UTC m=+33.248851405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:27:54.799965 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.799842 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:28:10.79983258 +0000 UTC m=+33.248884967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:27:54.900512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:54.900475 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:54.900668 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.900606 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:27:54.900668 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.900623 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:27:54.900668 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.900634 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:54.900805 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:54.900684 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:10.900671019 +0000 UTC m=+33.349723398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:27:55.068799 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:55.068728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:55.069121 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:55.068823 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:55.143808 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:55.143774 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a2566d6b667c14d54da2b5aaa448787" containerID="d92838ef4b5a2d1e594b2e787b08b77054d5e4b82e63f09c5377ef549388f3a9" exitCode=0 Jul 09 17:27:55.143951 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:55.143824 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" event={"ID":"3a2566d6b667c14d54da2b5aaa448787","Type":"ContainerDied","Data":"d92838ef4b5a2d1e594b2e787b08b77054d5e4b82e63f09c5377ef549388f3a9"} Jul 09 17:27:55.241908 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:55.241867 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:55.242662 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:55.242639 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:56.069135 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:56.069090 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:56.069692 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:56.069155 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:56.069692 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:56.069297 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:56.069692 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:56.069444 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:57.069030 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:57.068991 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:57.069205 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:57.069132 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:57.141262 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:57.141227 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:57.141872 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:57.141853 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-csfxj" Jul 09 17:27:58.072195 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:58.071280 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:27:58.072195 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:58.071539 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:27:58.072195 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:58.072040 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:27:58.072730 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:58.072207 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:27:59.068638 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:59.068606 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:27:59.068862 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:27:59.068732 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:27:59.603751 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:27:59.603294 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 17:28:00.068892 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.068812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:00.069043 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:00.068953 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:00.069043 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.069015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:00.069157 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:00.069121 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:00.073604 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.073526 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T17:27:59.603516311Z","UUID":"0dee1fd4-cf9c-4e18-978a-fe46badf6564","Handler":null,"Name":"","Endpoint":""} Jul 09 17:28:00.075061 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.075044 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 17:28:00.075142 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.075077 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 17:28:00.156406 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.156354 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="3c1ed9bed6a06e9dc56e5a854d86ed3c6b263c5fa5944c2c0e0f4e5568f86862" exitCode=0 Jul 09 17:28:00.156585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.156436 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"3c1ed9bed6a06e9dc56e5a854d86ed3c6b263c5fa5944c2c0e0f4e5568f86862"} Jul 09 17:28:00.159068 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.159039 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" event={"ID":"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95","Type":"ContainerStarted","Data":"30def3549cec13d54fbf0e8c10ca5f8a2aa210a5db4c873c1d684b6038c7a98e"} Jul 09 17:28:00.160929 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.160904 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9s5d" event={"ID":"047648cf-d30a-4556-aa48-5ba980836a26","Type":"ContainerStarted","Data":"13cd63d0630fd02cb0d90909534277467f4df3595592321321551d4d00537861"} Jul 09 17:28:00.163067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.163048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" event={"ID":"3a2566d6b667c14d54da2b5aaa448787","Type":"ContainerStarted","Data":"f875921dabb9a852a6078dbcf1e44802427e64844e44f3ba6a9e6516d32269d8"} Jul 09 17:28:00.166253 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"cd5908356fb37d7397ed5cd2fc99e88bb70cdf33a3b20c5105c8b6ecba4121b0"} Jul 09 17:28:00.166338 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166304 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"6debb3ea3d81009d78ba4fb686aedc92f45d65fce0b8d4d6cb8a2537902f170c"} Jul 09 17:28:00.166424 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166358 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"60e316d43b61ec9a45cdc45cd3347466439c1fb84073be85936b97986abf7622"} Jul 09 17:28:00.166482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166423 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"e2b702da17affb430c2d11b8ad528ef45a9e9dd8341fe44664ae4e27bb8cdf5c"} Jul 09 17:28:00.166482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"9fdf98862a2616f55b10debd2be18256da05983cb43671f0d403560ac929cf02"} Jul 09 17:28:00.166587 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.166517 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"f8521c068c607a60669f205ec8b609992bb1a61a139fa06217930ba453933625"} Jul 09 17:28:00.190195 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.190143 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n9s5d" podStartSLOduration=2.186191036 podStartE2EDuration="22.190128474s" totalImagesPullingTime="20.003937438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:00.18933457 +0000 UTC m=+22.638386996" watchObservedRunningTime="2026-07-09 17:28:00.190128474 +0000 UTC m=+22.639180875" Jul 09 17:28:00.202761 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:00.202712 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-12.ec2.internal" podStartSLOduration=22.20269679 podStartE2EDuration="22.20269679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:00.202158804 +0000 UTC m=+22.651211227" watchObservedRunningTime="2026-07-09 17:28:00.20269679 +0000 UTC m=+22.651749191" Jul 09 17:28:01.068926 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:01.068759 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:01.069486 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:01.068991 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:01.171388 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:01.171284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" event={"ID":"cdbfedda-d4b5-408b-9cfa-9a884b0a7a95","Type":"ContainerStarted","Data":"816a8c11fdde68a71b2ccf4c0d1a77bac575dc7baa2abd6da22f087485487633"} Jul 09 17:28:01.188104 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:01.188057 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xd2wg" podStartSLOduration=2.96621777 podStartE2EDuration="23.188043304s" totalImagesPullingTime="20.221825534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:01.187325398 +0000 UTC m=+23.636377799" watchObservedRunningTime="2026-07-09 17:28:01.188043304 +0000 UTC m=+23.637095724" Jul 09 17:28:02.069057 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:02.069017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:02.069568 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:02.069017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:02.069568 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:02.069176 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:02.069568 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:02.069229 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:03.068663 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:03.068634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:03.068909 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:03.068744 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:03.178615 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:03.178579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"fcba0f19523e3db3a931a128281293b737a2fed28a2e59b676a5122ea92fb0c5"} Jul 09 17:28:04.068523 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:04.068490 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:04.068707 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:04.068495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:04.068707 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:04.068624 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:04.068830 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:04.068712 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:05.068804 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.068758 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:05.069780 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:05.068899 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:05.187325 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.187285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" event={"ID":"a6b88fc1-61bd-40c3-8a51-0d4153c69782","Type":"ContainerStarted","Data":"75e798d00c348bff68ea1e230f84cc6f1194b46d0a95e6b289ea3b4f35462676"} Jul 09 17:28:05.187652 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.187625 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:05.187773 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.187699 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:05.205433 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.205401 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:05.212836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:05.212788 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" podStartSLOduration=7.189934036 podStartE2EDuration="27.212772069s" totalImagesPullingTime="20.022838033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:05.212182395 +0000 UTC m=+27.661234798" watchObservedRunningTime="2026-07-09 17:28:05.212772069 +0000 UTC m=+27.661824472" Jul 09 17:28:06.068877 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.068630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:06.069494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.068656 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:06.069494 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:06.068995 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:06.069494 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:06.069080 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:06.189621 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.189586 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:06.204914 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.204885 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:06.259338 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.259307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2dbb5"] Jul 09 17:28:06.259501 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.259443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:06.259564 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:06.259544 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:06.262463 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.262433 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bwx97"] Jul 09 17:28:06.262590 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.262531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:06.262660 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:06.262616 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:06.263103 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.263082 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t5g2n"] Jul 09 17:28:06.263187 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:06.263167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:06.263267 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:06.263251 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:07.193343 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:07.193309 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="c6e204c4e03ee1c3f123a77078b75b3a48e9b6f7a820429a5889a0ad2086d031" exitCode=0 Jul 09 17:28:07.193801 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:07.193400 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"c6e204c4e03ee1c3f123a77078b75b3a48e9b6f7a820429a5889a0ad2086d031"} Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:08.069701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:08.069796 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:08.069839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:08.069871 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:08.069912 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:08.070241 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:08.069985 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:09.199527 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:09.199437 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="cde011740138c5d7356ee9e85df727481807414759ebd04a1b70a31d80584fb8" exitCode=0 Jul 09 17:28:09.199527 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:09.199496 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"cde011740138c5d7356ee9e85df727481807414759ebd04a1b70a31d80584fb8"} Jul 09 17:28:10.068226 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.068190 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:10.068397 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.068233 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:10.068397 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.068190 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:10.068397 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.068311 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:10.068569 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.068413 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:10.068569 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.068498 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:10.810928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.810712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:10.810928 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.810899 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:28:10.811300 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.810971 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:28:10.811300 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.810977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs podName:431e5c77-b3ad-49b7-87d9-95ee5648429b nodeName:}" failed. No retries permitted until 2026-07-09 17:28:42.810957313 +0000 UTC m=+65.260009695 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs") pod "network-metrics-daemon-t5g2n" (UID: "431e5c77-b3ad-49b7-87d9-95ee5648429b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:28:10.811300 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.810903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:10.811300 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.811018 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret podName:21fe8f0d-338f-4886-bed2-a7ec244552ba nodeName:}" failed. No retries permitted until 2026-07-09 17:28:42.811002522 +0000 UTC m=+65.260054915 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret") pod "global-pull-secret-syncer-2dbb5" (UID: "21fe8f0d-338f-4886-bed2-a7ec244552ba") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:28:10.911987 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:10.911950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:10.912152 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.912079 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:28:10.912152 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.912095 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:28:10.912152 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.912103 2577 projected.go:196] Error preparing data for projected volume kube-api-access-cvhmk for pod openshift-network-diagnostics/network-check-target-bwx97: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:28:10.912253 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:10.912155 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk podName:e4ad64d2-3ae5-4d0f-967b-b2856971662f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:42.912139978 +0000 UTC m=+65.361192364 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cvhmk" (UniqueName: "kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk") pod "network-check-target-bwx97" (UID: "e4ad64d2-3ae5-4d0f-967b-b2856971662f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:28:11.205669 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:11.205633 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="e1c73c8b8199323ff1e876c44bcce13638beae3ccdec45c0454140337561d2a0" exitCode=0 Jul 09 17:28:11.205834 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:11.205696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"e1c73c8b8199323ff1e876c44bcce13638beae3ccdec45c0454140337561d2a0"} Jul 09 17:28:12.068848 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.068804 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:12.068848 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.068827 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:12.069352 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.068812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:12.069352 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.068925 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2dbb5" podUID="21fe8f0d-338f-4886-bed2-a7ec244552ba" Jul 09 17:28:12.069352 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.068989 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t5g2n" podUID="431e5c77-b3ad-49b7-87d9-95ee5648429b" Jul 09 17:28:12.069352 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.069067 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bwx97" podUID="e4ad64d2-3ae5-4d0f-967b-b2856971662f" Jul 09 17:28:12.336499 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.336464 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-12.ec2.internal" event="NodeReady" Jul 09 17:28:12.336686 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.336603 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 17:28:12.370257 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.370217 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc"] Jul 09 17:28:12.388197 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.388124 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q"] Jul 09 17:28:12.389207 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.389183 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.392337 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.391616 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 17:28:12.392337 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.391653 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cn56n\"" Jul 09 17:28:12.392337 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.391957 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.394680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.394655 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.395172 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.394936 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 17:28:12.406749 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.406723 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54d66b6fb6-lfbzr"] Jul 09 17:28:12.406901 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.406883 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.410163 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.409680 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 17:28:12.412962 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.412332 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mw8jm\"" Jul 09 17:28:12.412962 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.412601 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 17:28:12.413396 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.413182 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.415043 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.413744 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.428998 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.428969 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:28:12.429152 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.429105 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.431576 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.431549 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 17:28:12.431973 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.431950 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.432103 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.432012 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 17:28:12.432299 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.432280 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 17:28:12.432299 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.432294 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-m8drm\"" Jul 09 17:28:12.432429 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.432336 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 17:28:12.432471 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.432425 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.451166 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.451133 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-rq8nf"] Jul 09 17:28:12.451301 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.451264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.453582 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.453557 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 17:28:12.453717 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.453614 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 17:28:12.453717 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.453565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6m8jk\"" Jul 09 17:28:12.453826 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.453739 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 17:28:12.459204 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.459167 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 17:28:12.467140 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.467069 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt"] Jul 09 17:28:12.467280 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.467163 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.470327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.470304 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 17:28:12.470475 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.470452 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.470548 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.470328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 17:28:12.470598 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.470561 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.470768 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.470752 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dpkjn\"" Jul 09 17:28:12.475858 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.475812 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 17:28:12.484290 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.484261 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-66x4d"] Jul 09 17:28:12.484467 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.484394 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" Jul 09 17:28:12.487210 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.487178 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.487210 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.487196 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.487524 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.487507 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kbb7m\"" Jul 09 17:28:12.504597 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.504566 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg"] Jul 09 17:28:12.504760 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.504678 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.507103 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.507080 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 17:28:12.507253 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.507234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 17:28:12.507539 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.507520 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.507539 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.507532 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.508621 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.508352 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4cmmq\"" Jul 09 17:28:12.515557 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.515533 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 17:28:12.522340 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce3654b-14f1-431b-98a6-d82060ce7332-config\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.522494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.522494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6qt\" (UniqueName: \"kubernetes.io/projected/646b2559-7599-4b05-b2ab-fa37a13626fd-kube-api-access-6t6qt\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.522494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.522494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-config\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10d58688-da25-40ec-9950-986e594fd257-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522592 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqkr\" (UniqueName: \"kubernetes.io/projected/bce3654b-14f1-431b-98a6-d82060ce7332-kube-api-access-kqqkr\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522630 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-default-certificate\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.522754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl7x4\" (UniqueName: \"kubernetes.io/projected/10d58688-da25-40ec-9950-986e594fd257-kube-api-access-xl7x4\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-stats-auth\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpbbp\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/341cad21-7443-4c36-9c1e-29779538a458-serving-cert\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522971 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-trusted-ca\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.522996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce3654b-14f1-431b-98a6-d82060ce7332-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.523067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.523021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsthx\" (UniqueName: \"kubernetes.io/projected/341cad21-7443-4c36-9c1e-29779538a458-kube-api-access-wsthx\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.530764 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.530739 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2"] Jul 09 17:28:12.530898 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.530874 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.533440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.533419 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.533569 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.533491 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 17:28:12.533569 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.533514 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 17:28:12.533569 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.533534 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mnnk7\"" Jul 09 17:28:12.533569 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.533419 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.546805 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.546772 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc"] Jul 09 17:28:12.546805 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.546812 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q"] Jul 09 17:28:12.547003 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.546835 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt"] Jul 09 17:28:12.547003 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.546926 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lbnts"] Jul 09 17:28:12.547003 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.546940 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.549751 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.549596 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.549751 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.549611 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 17:28:12.549751 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.549614 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.549751 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.549601 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4gsbp\"" Jul 09 17:28:12.564026 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.563989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-66x4d"] Jul 09 17:28:12.564196 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.564043 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.564196 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.564084 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-prsrd"] Jul 09 17:28:12.566542 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.566519 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.566668 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.566528 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z9xhd\"" Jul 09 17:28:12.566668 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.566637 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.566774 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.566729 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 17:28:12.581699 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.581669 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9"] Jul 09 17:28:12.581862 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.581772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.584460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.584314 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 17:28:12.584460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.584331 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmgfn\"" Jul 09 17:28:12.584460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.584341 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 17:28:12.599899 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.599854 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg"] Jul 09 17:28:12.599899 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.599903 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5"] Jul 09 17:28:12.600214 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.599945 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.602427 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.602400 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 17:28:12.602559 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.602402 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5rvxx\"" Jul 09 17:28:12.602624 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.602578 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.602624 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.602584 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.602953 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.602906 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 17:28:12.622917 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.622877 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-rq8nf"] Jul 09 17:28:12.622917 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.622897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" Jul 09 17:28:12.623141 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.622934 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq"] Jul 09 17:28:12.623545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce3654b-14f1-431b-98a6-d82060ce7332-config\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.623682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.623682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623585 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-tmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.623682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-config\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.623682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqkr\" (UniqueName: \"kubernetes.io/projected/bce3654b-14f1-431b-98a6-d82060ce7332-kube-api-access-kqqkr\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.623682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-default-certificate\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpbbp\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-trusted-ca\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl7x4\" (UniqueName: \"kubernetes.io/projected/10d58688-da25-40ec-9950-986e594fd257-kube-api-access-xl7x4\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.623928 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.623927 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.623977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.123963215 +0000 UTC m=+35.573015595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.623996 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/341cad21-7443-4c36-9c1e-29779538a458-serving-cert\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6qt\" (UniqueName: \"kubernetes.io/projected/646b2559-7599-4b05-b2ab-fa37a13626fd-kube-api-access-6t6qt\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.624053 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624114 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bce3654b-14f1-431b-98a6-d82060ce7332-config\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.624068 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:12.624234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624217 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.624246 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.124230395 +0000 UTC m=+35.573282795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkmp\" (UniqueName: \"kubernetes.io/projected/bedd31c3-14f3-42f6-8da8-133b13e061e3-kube-api-access-tgkmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10d58688-da25-40ec-9950-986e594fd257-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624549 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-config\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.624696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624615 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.625009 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.624985 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10d58688-da25-40ec-9950-986e594fd257-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.625118 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.625118 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnxwr\" (UniqueName: \"kubernetes.io/projected/291fddd5-d527-4a82-a144-e5aefb8aef5c-kube-api-access-gnxwr\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.625118 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.625088 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.125073849 +0000 UTC m=+35.574126233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:12.625118 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.625101 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:12.625317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-snapshots\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.625317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625180 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bedd31c3-14f3-42f6-8da8-133b13e061e3-serving-cert\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.625317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625220 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q2sq\" (UniqueName: \"kubernetes.io/projected/41c280e1-4c69-4a64-895b-68aa2656f849-kube-api-access-7q2sq\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.625317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625254 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.625317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625294 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/341cad21-7443-4c36-9c1e-29779538a458-trusted-ca\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.625525 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.625388 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.125347706 +0000 UTC m=+35.574400096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:12.625633 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8hd9\" (UniqueName: \"kubernetes.io/projected/2f94944d-8636-4b03-8888-7f7c606524dc-kube-api-access-l8hd9\") pod \"volume-data-source-validator-78495597f6-x89wt\" (UID: \"2f94944d-8636-4b03-8888-7f7c606524dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" Jul 09 17:28:12.625703 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.625689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-stats-auth\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.627573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.626131 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.627573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.626360 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:28:12.627573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.626673 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:28:12.627573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.626861 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-c4tcv\"" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.628462 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp7d4\" (UniqueName: \"kubernetes.io/projected/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-kube-api-access-xp7d4\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.628510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce3654b-14f1-431b-98a6-d82060ce7332-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.628540 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsthx\" (UniqueName: \"kubernetes.io/projected/341cad21-7443-4c36-9c1e-29779538a458-kube-api-access-wsthx\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.628578 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41c280e1-4c69-4a64-895b-68aa2656f849-cert\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.629635 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.629648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.629661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/341cad21-7443-4c36-9c1e-29779538a458-serving-cert\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.629750 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.629742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-default-certificate\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.630294 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.630253 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-stats-auth\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.631660 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.631617 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bce3654b-14f1-431b-98a6-d82060ce7332-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.633685 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.632709 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqkr\" (UniqueName: \"kubernetes.io/projected/bce3654b-14f1-431b-98a6-d82060ce7332-kube-api-access-kqqkr\") pod \"service-ca-operator-7695fc6c8c-7jb9q\" (UID: \"bce3654b-14f1-431b-98a6-d82060ce7332\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.634925 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.634817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.634925 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.634859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl7x4\" (UniqueName: \"kubernetes.io/projected/10d58688-da25-40ec-9950-986e594fd257-kube-api-access-xl7x4\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:12.636444 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.635417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpbbp\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:12.636444 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.635424 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6qt\" (UniqueName: \"kubernetes.io/projected/646b2559-7599-4b05-b2ab-fa37a13626fd-kube-api-access-6t6qt\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:12.643127 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.643094 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsthx\" (UniqueName: \"kubernetes.io/projected/341cad21-7443-4c36-9c1e-29779538a458-kube-api-access-wsthx\") pod \"console-operator-7866497494-rq8nf\" (UID: \"341cad21-7443-4c36-9c1e-29779538a458\") " pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.649911 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.649871 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c"] Jul 09 17:28:12.650060 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.650021 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.652672 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.652650 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 17:28:12.667956 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.667922 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54d66b6fb6-lfbzr"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.667977 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.667999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668025 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lbnts"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668043 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668059 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-prsrd"] Jul 09 17:28:12.668084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668087 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5"] Jul 09 17:28:12.668327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668106 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c"] Jul 09 17:28:12.668327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668129 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq"] Jul 09 17:28:12.668327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.668295 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.687389 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.685918 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 17:28:12.687389 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.686176 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 17:28:12.687389 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.686384 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 17:28:12.687389 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.686582 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 17:28:12.724649 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.724612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" Jul 09 17:28:12.729960 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.729928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h26sw\" (UniqueName: \"kubernetes.io/projected/03c4b441-78be-49f8-8090-d0fadd5af852-kube-api-access-h26sw\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.730117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.729966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q862g\" (UniqueName: \"kubernetes.io/projected/b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3-kube-api-access-q862g\") pod \"network-check-source-869dcfcdcd-k5td5\" (UID: \"b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" Jul 09 17:28:12.730117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.729995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c4b441-78be-49f8-8090-d0fadd5af852-tmp-dir\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.730117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.730117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55flh\" (UniqueName: \"kubernetes.io/projected/77588946-08cd-4216-8206-6b718aa3e4e2-kube-api-access-55flh\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.730117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrspq\" (UniqueName: \"kubernetes.io/projected/a9b1eb6d-cfc1-44d5-bc87-542da971c482-kube-api-access-hrspq\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730123 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9b1eb6d-cfc1-44d5-bc87-542da971c482-klusterlet-config\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.730283 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:12.730305 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730290 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.730680 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.730380 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.230343691 +0000 UTC m=+35.679396078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:12.730680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e10eb48-c9df-4f4f-a846-6dfef3928994-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.730680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.730680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.730680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.730912 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730812 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkmp\" (UniqueName: \"kubernetes.io/projected/bedd31c3-14f3-42f6-8da8-133b13e061e3-kube-api-access-tgkmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.730912 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnxwr\" (UniqueName: \"kubernetes.io/projected/291fddd5-d527-4a82-a144-e5aefb8aef5c-kube-api-access-gnxwr\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.731004 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.730986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-snapshots\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731055 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bedd31c3-14f3-42f6-8da8-133b13e061e3-serving-cert\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731099 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q2sq\" (UniqueName: \"kubernetes.io/projected/41c280e1-4c69-4a64-895b-68aa2656f849-kube-api-access-7q2sq\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.731099 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77588946-08cd-4216-8206-6b718aa3e4e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.731225 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8hd9\" (UniqueName: \"kubernetes.io/projected/2f94944d-8636-4b03-8888-7f7c606524dc-kube-api-access-l8hd9\") pod \"volume-data-source-validator-78495597f6-x89wt\" (UID: \"2f94944d-8636-4b03-8888-7f7c606524dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" Jul 09 17:28:12.731225 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp7d4\" (UniqueName: \"kubernetes.io/projected/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-kube-api-access-xp7d4\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.731324 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731324 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.731479 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b1eb6d-cfc1-44d5-bc87-542da971c482-tmp\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.731479 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69x2s\" (UniqueName: \"kubernetes.io/projected/2e10eb48-c9df-4f4f-a846-6dfef3928994-kube-api-access-69x2s\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.731479 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bedd31c3-14f3-42f6-8da8-133b13e061e3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41c280e1-4c69-4a64-895b-68aa2656f849-cert\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.731613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-tmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/03c4b441-78be-49f8-8090-d0fadd5af852-config-volume\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.731706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-snapshots\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.731881 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.731861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bedd31c3-14f3-42f6-8da8-133b13e061e3-tmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.733468 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.733440 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.733755 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.733734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bedd31c3-14f3-42f6-8da8-133b13e061e3-serving-cert\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.734482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.734460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41c280e1-4c69-4a64-895b-68aa2656f849-cert\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.735097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.735075 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.739867 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.739821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp7d4\" (UniqueName: \"kubernetes.io/projected/d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b-kube-api-access-xp7d4\") pod \"kube-storage-version-migrator-operator-799cb7f958-mwdxg\" (UID: \"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.740443 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.740399 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkmp\" (UniqueName: \"kubernetes.io/projected/bedd31c3-14f3-42f6-8da8-133b13e061e3-kube-api-access-tgkmp\") pod \"insights-operator-68d9dd94cf-66x4d\" (UID: \"bedd31c3-14f3-42f6-8da8-133b13e061e3\") " pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.742350 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.742308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnxwr\" (UniqueName: \"kubernetes.io/projected/291fddd5-d527-4a82-a144-e5aefb8aef5c-kube-api-access-gnxwr\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:12.742458 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.742357 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q2sq\" (UniqueName: \"kubernetes.io/projected/41c280e1-4c69-4a64-895b-68aa2656f849-kube-api-access-7q2sq\") pod \"ingress-canary-lbnts\" (UID: \"41c280e1-4c69-4a64-895b-68aa2656f849\") " pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.743129 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.743105 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8hd9\" (UniqueName: \"kubernetes.io/projected/2f94944d-8636-4b03-8888-7f7c606524dc-kube-api-access-l8hd9\") pod \"volume-data-source-validator-78495597f6-x89wt\" (UID: \"2f94944d-8636-4b03-8888-7f7c606524dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" Jul 09 17:28:12.777713 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.777683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:12.794972 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.794941 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" Jul 09 17:28:12.815759 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.815732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" Jul 09 17:28:12.832993 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.832876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/03c4b441-78be-49f8-8090-d0fadd5af852-config-volume\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.832993 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.832926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h26sw\" (UniqueName: \"kubernetes.io/projected/03c4b441-78be-49f8-8090-d0fadd5af852-kube-api-access-h26sw\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.832993 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.832957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q862g\" (UniqueName: \"kubernetes.io/projected/b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3-kube-api-access-q862g\") pod \"network-check-source-869dcfcdcd-k5td5\" (UID: \"b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" Jul 09 17:28:12.832993 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.832989 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c4b441-78be-49f8-8090-d0fadd5af852-tmp-dir\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55flh\" (UniqueName: \"kubernetes.io/projected/77588946-08cd-4216-8206-6b718aa3e4e2-kube-api-access-55flh\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrspq\" (UniqueName: \"kubernetes.io/projected/a9b1eb6d-cfc1-44d5-bc87-542da971c482-kube-api-access-hrspq\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833094 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9b1eb6d-cfc1-44d5-bc87-542da971c482-klusterlet-config\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e10eb48-c9df-4f4f-a846-6dfef3928994-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.833289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.833647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833390 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77588946-08cd-4216-8206-6b718aa3e4e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.833647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b1eb6d-cfc1-44d5-bc87-542da971c482-tmp\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.833647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.833506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69x2s\" (UniqueName: \"kubernetes.io/projected/2e10eb48-c9df-4f4f-a846-6dfef3928994-kube-api-access-69x2s\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.835971 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.834783 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c4b441-78be-49f8-8090-d0fadd5af852-tmp-dir\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.835971 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.835760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/03c4b441-78be-49f8-8090-d0fadd5af852-config-volume\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.837095 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.836763 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:12.837095 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:12.836891 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:13.336862122 +0000 UTC m=+35.785914521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:12.837095 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.836791 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b1eb6d-cfc1-44d5-bc87-542da971c482-tmp\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.837979 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.837912 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77588946-08cd-4216-8206-6b718aa3e4e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.839201 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.839126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-ca\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.840724 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.840700 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a9b1eb6d-cfc1-44d5-bc87-542da971c482-klusterlet-config\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.841513 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.840998 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" Jul 09 17:28:12.841513 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.841034 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e10eb48-c9df-4f4f-a846-6dfef3928994-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.845334 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.844937 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.853049 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.852988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h26sw\" (UniqueName: \"kubernetes.io/projected/03c4b441-78be-49f8-8090-d0fadd5af852-kube-api-access-h26sw\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:12.853220 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.853192 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrspq\" (UniqueName: \"kubernetes.io/projected/a9b1eb6d-cfc1-44d5-bc87-542da971c482-kube-api-access-hrspq\") pod \"klusterlet-addon-workmgr-777df645c6-zswhq\" (UID: \"a9b1eb6d-cfc1-44d5-bc87-542da971c482\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.854306 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.853916 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.854306 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.854129 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69x2s\" (UniqueName: \"kubernetes.io/projected/2e10eb48-c9df-4f4f-a846-6dfef3928994-kube-api-access-69x2s\") pod \"managed-serviceaccount-addon-agent-85656d8758-rbmq9\" (UID: \"2e10eb48-c9df-4f4f-a846-6dfef3928994\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.854306 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.854262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q862g\" (UniqueName: \"kubernetes.io/projected/b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3-kube-api-access-q862g\") pod \"network-check-source-869dcfcdcd-k5td5\" (UID: \"b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" Jul 09 17:28:12.854775 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.854749 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77588946-08cd-4216-8206-6b718aa3e4e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.855358 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.855312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55flh\" (UniqueName: \"kubernetes.io/projected/77588946-08cd-4216-8206-6b718aa3e4e2-kube-api-access-55flh\") pod \"cluster-proxy-proxy-agent-79c4779f69-rt96c\" (UID: \"77588946-08cd-4216-8206-6b718aa3e4e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:12.874536 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.874471 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lbnts" Jul 09 17:28:12.924741 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.924473 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" Jul 09 17:28:12.944423 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.944326 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q"] Jul 09 17:28:12.962093 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.961899 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" Jul 09 17:28:12.971426 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.971093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:12.998769 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:12.998570 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:28:13.012946 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.012813 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-rq8nf"] Jul 09 17:28:13.018520 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.018478 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341cad21_7443_4c36_9c1e_29779538a458.slice/crio-bd14f108c637bf179acb8243b1403507acfd9ff0075e00cb917e4a4d25ec874f WatchSource:0}: Error finding container bd14f108c637bf179acb8243b1403507acfd9ff0075e00cb917e4a4d25ec874f: Status 404 returned error can't find the container with id bd14f108c637bf179acb8243b1403507acfd9ff0075e00cb917e4a4d25ec874f Jul 09 17:28:13.031390 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.031343 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt"] Jul 09 17:28:13.045613 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.045563 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f94944d_8636_4b03_8888_7f7c606524dc.slice/crio-edb1a4f63f52f689dbd8778de760f5772e664e828f74cfb8066b17baf9971515 WatchSource:0}: Error finding container edb1a4f63f52f689dbd8778de760f5772e664e828f74cfb8066b17baf9971515: Status 404 returned error can't find the container with id edb1a4f63f52f689dbd8778de760f5772e664e828f74cfb8066b17baf9971515 Jul 09 17:28:13.053429 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.053347 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-66x4d"] Jul 09 17:28:13.068480 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.068078 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbedd31c3_14f3_42f6_8da8_133b13e061e3.slice/crio-546c97facc5ef269e31f8a147702697b7e00ff6cd86038783350bf92f0213e97 WatchSource:0}: Error finding container 546c97facc5ef269e31f8a147702697b7e00ff6cd86038783350bf92f0213e97: Status 404 returned error can't find the container with id 546c97facc5ef269e31f8a147702697b7e00ff6cd86038783350bf92f0213e97 Jul 09 17:28:13.080277 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.079581 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg"] Jul 09 17:28:13.084871 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.084842 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd876a62a_e6e9_4d40_8ae7_8ed4bcf7910b.slice/crio-4f7dacbfe52530e68e73902e8905cfdaca60f3e2a642720b66676d49fa7dfe35 WatchSource:0}: Error finding container 4f7dacbfe52530e68e73902e8905cfdaca60f3e2a642720b66676d49fa7dfe35: Status 404 returned error can't find the container with id 4f7dacbfe52530e68e73902e8905cfdaca60f3e2a642720b66676d49fa7dfe35 Jul 09 17:28:13.115185 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.115109 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lbnts"] Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.144000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.144041 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.144072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.144132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144287 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144347 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.144328345 +0000 UTC m=+36.593380741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144443 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144482 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.14446877 +0000 UTC m=+36.593521154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144549 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144563 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144594 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.144583327 +0000 UTC m=+36.593635709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:13.144613 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.144620 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.144609399 +0000 UTC m=+36.593661790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:13.181494 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.181467 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5"] Jul 09 17:28:13.184545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.184508 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9"] Jul 09 17:28:13.185149 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.185120 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ce6c69_1817_4bf3_bbf3_3b2adf6036e3.slice/crio-60b39e9b20ff4d8e6aa702dac511166a0e7e4a5055bf2a1a234063381cfc19ef WatchSource:0}: Error finding container 60b39e9b20ff4d8e6aa702dac511166a0e7e4a5055bf2a1a234063381cfc19ef: Status 404 returned error can't find the container with id 60b39e9b20ff4d8e6aa702dac511166a0e7e4a5055bf2a1a234063381cfc19ef Jul 09 17:28:13.189436 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.189303 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e10eb48_c9df_4f4f_a846_6dfef3928994.slice/crio-021fd566c3a887af2f181f58a391f30326d8a47cf8afb04a995b654a955aa1a8 WatchSource:0}: Error finding container 021fd566c3a887af2f181f58a391f30326d8a47cf8afb04a995b654a955aa1a8: Status 404 returned error can't find the container with id 021fd566c3a887af2f181f58a391f30326d8a47cf8afb04a995b654a955aa1a8 Jul 09 17:28:13.212210 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.212167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" event={"ID":"bedd31c3-14f3-42f6-8da8-133b13e061e3","Type":"ContainerStarted","Data":"546c97facc5ef269e31f8a147702697b7e00ff6cd86038783350bf92f0213e97"} Jul 09 17:28:13.213651 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.213622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rq8nf" event={"ID":"341cad21-7443-4c36-9c1e-29779538a458","Type":"ContainerStarted","Data":"bd14f108c637bf179acb8243b1403507acfd9ff0075e00cb917e4a4d25ec874f"} Jul 09 17:28:13.214895 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.214857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" event={"ID":"2e10eb48-c9df-4f4f-a846-6dfef3928994","Type":"ContainerStarted","Data":"021fd566c3a887af2f181f58a391f30326d8a47cf8afb04a995b654a955aa1a8"} Jul 09 17:28:13.216512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.216341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" event={"ID":"bce3654b-14f1-431b-98a6-d82060ce7332","Type":"ContainerStarted","Data":"eb9199001793e66384b92b4e21d71febc295ebf1fb40d8f863a3edf889aa8dbd"} Jul 09 17:28:13.217987 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.217942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" event={"ID":"b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3","Type":"ContainerStarted","Data":"60b39e9b20ff4d8e6aa702dac511166a0e7e4a5055bf2a1a234063381cfc19ef"} Jul 09 17:28:13.219148 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.219122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lbnts" event={"ID":"41c280e1-4c69-4a64-895b-68aa2656f849","Type":"ContainerStarted","Data":"2e183cd7068e0fa75772f3d491372ba965463fb2b14a604b1882bed5e9502538"} Jul 09 17:28:13.220346 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.220300 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" event={"ID":"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b","Type":"ContainerStarted","Data":"4f7dacbfe52530e68e73902e8905cfdaca60f3e2a642720b66676d49fa7dfe35"} Jul 09 17:28:13.222120 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.222100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" event={"ID":"2f94944d-8636-4b03-8888-7f7c606524dc","Type":"ContainerStarted","Data":"edb1a4f63f52f689dbd8778de760f5772e664e828f74cfb8066b17baf9971515"} Jul 09 17:28:13.229648 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.229530 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq"] Jul 09 17:28:13.232346 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.232319 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b1eb6d_cfc1_44d5_bc87_542da971c482.slice/crio-15bdc5a079049d2048efc326f3c409f7a57c46cf6abbdfe6a3e26fd8e2e67f59 WatchSource:0}: Error finding container 15bdc5a079049d2048efc326f3c409f7a57c46cf6abbdfe6a3e26fd8e2e67f59: Status 404 returned error can't find the container with id 15bdc5a079049d2048efc326f3c409f7a57c46cf6abbdfe6a3e26fd8e2e67f59 Jul 09 17:28:13.238387 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.238348 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c"] Jul 09 17:28:13.240658 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:13.240638 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77588946_08cd_4216_8206_6b718aa3e4e2.slice/crio-348c398922e39d60c7f9ec1c64f6c0f7ff96ce5b3a085d60548b1648cf244b94 WatchSource:0}: Error finding container 348c398922e39d60c7f9ec1c64f6c0f7ff96ce5b3a085d60548b1648cf244b94: Status 404 returned error can't find the container with id 348c398922e39d60c7f9ec1c64f6c0f7ff96ce5b3a085d60548b1648cf244b94 Jul 09 17:28:13.245342 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.245318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:13.245488 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.245468 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:13.245551 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.245540 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.24551846 +0000 UTC m=+36.694570854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:13.346214 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:13.346127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:13.346388 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.346269 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:13.346388 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:13.346344 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:14.346323341 +0000 UTC m=+36.795375724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:14.072348 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.071115 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:14.072348 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.071478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:14.072348 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.072029 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:14.074674 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.074337 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:28:14.074674 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.074429 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ls9tf\"" Jul 09 17:28:14.078319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.077578 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:28:14.078319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.077863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8889q\"" Jul 09 17:28:14.153283 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.153235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:14.153744 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.153293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:14.153744 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.153322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:14.153744 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.153395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:14.153744 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.153653 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:14.153942 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.153764 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.153744018 +0000 UTC m=+38.602796405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:14.154230 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154212 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:14.154327 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154268 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.154253209 +0000 UTC m=+38.603305589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:14.154422 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154341 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:14.154422 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154353 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:14.154422 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154405 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.154392529 +0000 UTC m=+38.603444912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:14.154585 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.154481 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.15447064 +0000 UTC m=+38.603523026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:14.245997 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.245958 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerStarted","Data":"348c398922e39d60c7f9ec1c64f6c0f7ff96ce5b3a085d60548b1648cf244b94"} Jul 09 17:28:14.248440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.248397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" event={"ID":"a9b1eb6d-cfc1-44d5-bc87-542da971c482","Type":"ContainerStarted","Data":"15bdc5a079049d2048efc326f3c409f7a57c46cf6abbdfe6a3e26fd8e2e67f59"} Jul 09 17:28:14.254505 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.254476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:14.254737 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.254721 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:14.254812 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.254786 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.254768147 +0000 UTC m=+38.703820531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:14.356096 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:14.355523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:14.356096 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.355685 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:14.356096 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:14.355747 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:16.355728703 +0000 UTC m=+38.804781089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.175943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.175998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.176032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.176095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176238 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176299 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.176281313 +0000 UTC m=+42.625333700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176711 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176764 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.176746879 +0000 UTC m=+42.625799265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176827 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176837 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176869 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.176859273 +0000 UTC m=+42.625911668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:16.176978 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.176937 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.17692556 +0000 UTC m=+42.625977945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:16.276752 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.276714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:16.277000 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.276976 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:16.277119 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.277049 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.277030416 +0000 UTC m=+42.726082809 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:16.377903 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:16.377860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:16.378226 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.378171 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:16.378329 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:16.378252 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:20.378234127 +0000 UTC m=+42.827286506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:20.218920 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.218868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:20.218920 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.218924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.218953 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219033 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.219050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219109 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.219086449 +0000 UTC m=+50.668138844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219130 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.219118907 +0000 UTC m=+50.668171294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219149 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219162 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219181 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219203 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.219187976 +0000 UTC m=+50.668240356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:20.219475 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.219228 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.219218674 +0000 UTC m=+50.668271054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:20.319979 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.319934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:20.320165 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.320107 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:20.320224 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.320185 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.320163496 +0000 UTC m=+50.769215878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:20.420635 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:20.420598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:20.420882 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.420783 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:20.420882 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:20.420863 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:28.420844063 +0000 UTC m=+50.869896464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:22.162554 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.162520 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jjwkq"] Jul 09 17:28:22.181296 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.181259 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jjwkq"] Jul 09 17:28:22.181471 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.181420 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.184320 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.184292 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 17:28:22.184320 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.184312 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 17:28:22.184536 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.184418 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5q8ns\"" Jul 09 17:28:22.339650 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.339614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.339823 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.339753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.441082 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.440952 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.441248 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:22.441132 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:22.441318 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:22.441255 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:22.941233416 +0000 UTC m=+45.390285802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:22.441318 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.441286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.442079 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.442053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.946533 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:22.946493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:22.946696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:22.946609 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:22.946696 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:22.946669 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:23.94665571 +0000 UTC m=+46.395708091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:23.956252 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:23.956216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:23.956624 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:23.956408 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:23.956624 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:23.956487 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:25.956471911 +0000 UTC m=+48.405524290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:25.976571 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:25.976519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:25.977011 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:25.976680 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:25.977011 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:25.976752 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:29.976733201 +0000 UTC m=+52.425785588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:28.301577 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.301547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:28.301963 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.301645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:28.301963 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.301666 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:28.301963 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.301691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:28.301963 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.301802 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.301789261 +0000 UTC m=+66.750841641 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : configmap references non-existent config key: service-ca.crt Jul 09 17:28:28.302196 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302179 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:28:28.302390 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302357 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs podName:646b2559-7599-4b05-b2ab-fa37a13626fd nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.30234074 +0000 UTC m=+66.751393122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs") pod "router-default-54d66b6fb6-lfbzr" (UID: "646b2559-7599-4b05-b2ab-fa37a13626fd") : secret "router-metrics-certs-default" not found Jul 09 17:28:28.302528 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302257 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:28.302647 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302266 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:28:28.302707 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302652 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8575c6d767-6cxr9: secret "image-registry-tls" not found Jul 09 17:28:28.302770 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302758 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls podName:10d58688-da25-40ec-9950-986e594fd257 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.302629229 +0000 UTC m=+66.751681612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jfltc" (UID: "10d58688-da25-40ec-9950-986e594fd257") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:28:28.302871 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.302861 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls podName:f36d114c-d106-482d-9137-57aad81d6975 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.302847198 +0000 UTC m=+66.751899587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls") pod "image-registry-8575c6d767-6cxr9" (UID: "f36d114c-d106-482d-9137-57aad81d6975") : secret "image-registry-tls" not found Jul 09 17:28:28.408472 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.408124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:28.408472 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.408383 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:28:28.408472 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.408438 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls podName:291fddd5-d527-4a82-a144-e5aefb8aef5c nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.408421344 +0000 UTC m=+66.857473738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls") pod "cluster-samples-operator-65db858894-857c2" (UID: "291fddd5-d527-4a82-a144-e5aefb8aef5c") : secret "samples-operator-tls" not found Jul 09 17:28:28.509960 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:28.509394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:28.509960 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.509558 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:28:28.509960 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:28.509615 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls podName:03c4b441-78be-49f8-8090-d0fadd5af852 nodeName:}" failed. No retries permitted until 2026-07-09 17:28:44.509598345 +0000 UTC m=+66.958650730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls") pod "dns-default-prsrd" (UID: "03c4b441-78be-49f8-8090-d0fadd5af852") : secret "dns-default-metrics-tls" not found Jul 09 17:28:29.314853 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.314577 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" event={"ID":"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b","Type":"ContainerStarted","Data":"65ec7610f67325bcfbd16acaff898c3e45fef24622433d9dc16386ec2bf450d5"} Jul 09 17:28:29.316332 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.316301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" event={"ID":"2f94944d-8636-4b03-8888-7f7c606524dc","Type":"ContainerStarted","Data":"8fd6558fc327f4970efa2c48947a1d7884cdaff670294fd92b8e08a0157952d4"} Jul 09 17:28:29.319467 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.319440 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="42dcb73af9e8f6c3bdac9dc259d21b705911f9a95453afe7c12f10957292aa27" exitCode=0 Jul 09 17:28:29.319621 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.319521 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"42dcb73af9e8f6c3bdac9dc259d21b705911f9a95453afe7c12f10957292aa27"} Jul 09 17:28:29.321226 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.321204 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" event={"ID":"bedd31c3-14f3-42f6-8da8-133b13e061e3","Type":"ContainerStarted","Data":"2290a7aad68b4db250b76c585cac57050d19e68546cb2015d0d20bc003dfa70c"} Jul 09 17:28:29.323358 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.323336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/0.log" Jul 09 17:28:29.323483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.323392 2577 generic.go:356] "Generic (PLEG): container finished" podID="341cad21-7443-4c36-9c1e-29779538a458" containerID="94a8c67232098a478412b6096054f82773fddaacf7e465c09f9b3767e5cb4f0b" exitCode=255 Jul 09 17:28:29.323483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.323451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rq8nf" event={"ID":"341cad21-7443-4c36-9c1e-29779538a458","Type":"ContainerDied","Data":"94a8c67232098a478412b6096054f82773fddaacf7e465c09f9b3767e5cb4f0b"} Jul 09 17:28:29.323730 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.323714 2577 scope.go:117] "RemoveContainer" containerID="94a8c67232098a478412b6096054f82773fddaacf7e465c09f9b3767e5cb4f0b" Jul 09 17:28:29.325178 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.325159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" event={"ID":"2e10eb48-c9df-4f4f-a846-6dfef3928994","Type":"ContainerStarted","Data":"57b8125e3d90fe165722420cd2b75d63afee17de8fac35f572a443044a5191ce"} Jul 09 17:28:29.326940 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.326919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" event={"ID":"bce3654b-14f1-431b-98a6-d82060ce7332","Type":"ContainerStarted","Data":"7c97ed55246bd11c096233b39bf4c41c4e648c6bab536bdb5260ab01bab6bde5"} Jul 09 17:28:29.329723 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.329702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerStarted","Data":"3fc0c0c88ddf008c2d3a1ca237a110dd2c3e04b2ffa24daf6531aae13f8a19f5"} Jul 09 17:28:29.329804 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.329738 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerStarted","Data":"182818772aff3b26f2ec891473563d4adb2c6b05a5346620b92e9b94a2752248"} Jul 09 17:28:29.329804 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.329757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerStarted","Data":"e90255861d71d9f0624bf282961fc731200dc1f82f0fa6da5a85e6ae592ea5b4"} Jul 09 17:28:29.333140 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.333100 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" podStartSLOduration=25.264264948 podStartE2EDuration="40.33308135s" totalImagesPullingTime="15.068816402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.331222964 +0000 UTC m=+51.780275368" watchObservedRunningTime="2026-07-09 17:28:29.33308135 +0000 UTC m=+51.782133753" Jul 09 17:28:29.333295 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.333275 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" event={"ID":"a9b1eb6d-cfc1-44d5-bc87-542da971c482","Type":"ContainerStarted","Data":"ac3129ef4f785872f884127883c749929fee7dc77b91eecc950e17c7670a4d72"} Jul 09 17:28:29.334528 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.334501 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:29.335720 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.335702 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" Jul 09 17:28:29.336629 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.336604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" event={"ID":"b2ce6c69-1817-4bf3-bbf3-3b2adf6036e3","Type":"ContainerStarted","Data":"5bbf1b5847d6588539c4161f1e2e50c3f4e1102f830feb0a2462e4ff5476a487"} Jul 09 17:28:29.338324 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.338302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lbnts" event={"ID":"41c280e1-4c69-4a64-895b-68aa2656f849","Type":"ContainerStarted","Data":"161678dace8fc599af0ab9d5d20328ee6ee4b7f34acad72271592bba90fc6037"} Jul 09 17:28:29.387428 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.384558 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85656d8758-rbmq9" podStartSLOduration=19.410630215 podStartE2EDuration="34.384525191s" totalImagesPullingTime="14.973894976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.381631095 +0000 UTC m=+51.830683494" watchObservedRunningTime="2026-07-09 17:28:29.384525191 +0000 UTC m=+51.833577575" Jul 09 17:28:29.422567 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.421985 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" podStartSLOduration=25.21898879 podStartE2EDuration="40.421963203s" totalImagesPullingTime="15.202974413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.404328852 +0000 UTC m=+51.853381255" watchObservedRunningTime="2026-07-09 17:28:29.421963203 +0000 UTC m=+51.871015606" Jul 09 17:28:29.438292 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.438209 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" podStartSLOduration=25.382366485 podStartE2EDuration="40.438189927s" totalImagesPullingTime="15.055823442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.437530368 +0000 UTC m=+51.886582807" watchObservedRunningTime="2026-07-09 17:28:29.438189927 +0000 UTC m=+51.887242332" Jul 09 17:28:29.454787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.454600 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x89wt" podStartSLOduration=25.371134061 podStartE2EDuration="40.45457962s" totalImagesPullingTime="15.083445559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.453289452 +0000 UTC m=+51.902341851" watchObservedRunningTime="2026-07-09 17:28:29.45457962 +0000 UTC m=+51.903632022" Jul 09 17:28:29.473348 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.473298 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" podStartSLOduration=19.538299948 podStartE2EDuration="34.473276868s" totalImagesPullingTime="14.93497692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.472099458 +0000 UTC m=+51.921151863" watchObservedRunningTime="2026-07-09 17:28:29.473276868 +0000 UTC m=+51.922329260" Jul 09 17:28:29.508020 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.507202 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-777df645c6-zswhq" podStartSLOduration=19.524099497 podStartE2EDuration="34.50717994s" totalImagesPullingTime="14.983080443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.488149111 +0000 UTC m=+51.937201514" watchObservedRunningTime="2026-07-09 17:28:29.50717994 +0000 UTC m=+51.956232346" Jul 09 17:28:29.527668 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.527615 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-k5td5" podStartSLOduration=22.551955183 podStartE2EDuration="37.5275957s" totalImagesPullingTime="14.975640517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.526395422 +0000 UTC m=+51.975447851" watchObservedRunningTime="2026-07-09 17:28:29.5275957 +0000 UTC m=+51.976648103" Jul 09 17:28:29.528151 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.528119 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lbnts" podStartSLOduration=2.507662684 podStartE2EDuration="17.528108876s" totalImagesPullingTime="15.020446192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:29.508112548 +0000 UTC m=+51.957164952" watchObservedRunningTime="2026-07-09 17:28:29.528108876 +0000 UTC m=+51.977161280" Jul 09 17:28:29.878330 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.876773 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fjkw7"] Jul 09 17:28:29.880569 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.880547 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.883191 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.882947 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 17:28:29.883191 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.882978 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gxngq\"" Jul 09 17:28:29.883359 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.883208 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 17:28:29.892410 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.892359 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjkw7"] Jul 09 17:28:29.923770 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.923725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69fvh\" (UniqueName: \"kubernetes.io/projected/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-api-access-69fvh\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.923770 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.923771 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-data-volume\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.924022 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.923845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.924022 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.923975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.924139 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.924044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-crio-socket\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:29.970787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.970760 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4"] Jul 09 17:28:29.973959 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.973933 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" Jul 09 17:28:29.976321 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.976292 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6k8c4\"" Jul 09 17:28:29.976457 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.976341 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 17:28:29.976647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.976627 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 17:28:29.983031 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:29.983009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4"] Jul 09 17:28:30.024463 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-crio-socket\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024533 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024600 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:30.524580372 +0000 UTC m=+52.973632777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : configmap "kube-rbac-proxy" not found Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69fvh\" (UniqueName: \"kubernetes.io/projected/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-api-access-69fvh\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024639 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-crio-socket\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-data-volume\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024668 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.024727 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024714 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:38.024702936 +0000 UTC m=+60.473755328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:30.025310 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024792 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:28:30.025310 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlj64\" (UniqueName: \"kubernetes.io/projected/7dc70c29-19e7-40f1-a41a-9bcc21f33bf2-kube-api-access-zlj64\") pod \"migrator-7f59b54f6c-xn2m4\" (UID: \"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" Jul 09 17:28:30.025310 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.024840 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:30.524823008 +0000 UTC m=+52.973875406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : secret "insights-runtime-extractor-tls" not found Jul 09 17:28:30.025310 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.024962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-data-volume\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.038464 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.038428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69fvh\" (UniqueName: \"kubernetes.io/projected/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-api-access-69fvh\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.125848 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.125808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlj64\" (UniqueName: \"kubernetes.io/projected/7dc70c29-19e7-40f1-a41a-9bcc21f33bf2-kube-api-access-zlj64\") pod \"migrator-7f59b54f6c-xn2m4\" (UID: \"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" Jul 09 17:28:30.133108 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.133047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlj64\" (UniqueName: \"kubernetes.io/projected/7dc70c29-19e7-40f1-a41a-9bcc21f33bf2-kube-api-access-zlj64\") pod \"migrator-7f59b54f6c-xn2m4\" (UID: \"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" Jul 09 17:28:30.284599 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.284562 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" Jul 09 17:28:30.355001 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.354966 2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ce5-02b2-49ad-ab0e-c6d1a914f299" containerID="18a9297ba7256a63385acd2da47a23c4280c11d42a3cad67e200e47ff64781ab" exitCode=0 Jul 09 17:28:30.355459 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.355033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerDied","Data":"18a9297ba7256a63385acd2da47a23c4280c11d42a3cad67e200e47ff64781ab"} Jul 09 17:28:30.356729 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.356710 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:28:30.357110 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.357091 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/0.log" Jul 09 17:28:30.357162 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.357131 2577 generic.go:356] "Generic (PLEG): container finished" podID="341cad21-7443-4c36-9c1e-29779538a458" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" exitCode=255 Jul 09 17:28:30.357348 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.357325 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rq8nf" event={"ID":"341cad21-7443-4c36-9c1e-29779538a458","Type":"ContainerDied","Data":"b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d"} Jul 09 17:28:30.357445 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.357395 2577 scope.go:117] "RemoveContainer" containerID="94a8c67232098a478412b6096054f82773fddaacf7e465c09f9b3767e5cb4f0b" Jul 09 17:28:30.358143 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.358124 2577 scope.go:117] "RemoveContainer" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" Jul 09 17:28:30.358299 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.358278 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rq8nf_openshift-console-operator(341cad21-7443-4c36-9c1e-29779538a458)\"" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podUID="341cad21-7443-4c36-9c1e-29779538a458" Jul 09 17:28:30.427595 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.427572 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4"] Jul 09 17:28:30.532175 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.532132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.532343 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.532246 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:28:30.532343 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.532312 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:31.532297353 +0000 UTC m=+53.981349733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : configmap "kube-rbac-proxy" not found Jul 09 17:28:30.532343 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:30.532335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:30.532537 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.532527 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:28:30.532589 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:30.532584 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:31.53256737 +0000 UTC m=+53.981619754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : secret "insights-runtime-extractor-tls" not found Jul 09 17:28:31.070536 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.070510 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sjr8l_b83651f0-ec52-46a6-9671-24fef94d000b/dns-node-resolver/0.log" Jul 09 17:28:31.362513 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.362480 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" event={"ID":"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2","Type":"ContainerStarted","Data":"5de08e330a0c0829c893908af328e4438be8e93de58e6c1d395223e06c0a78cb"} Jul 09 17:28:31.366334 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.366305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bfldg" event={"ID":"3ad28ce5-02b2-49ad-ab0e-c6d1a914f299","Type":"ContainerStarted","Data":"3568b1afc7fcac7333ae775a7921d71ab836acba8d00e62b097c8264f71c8137"} Jul 09 17:28:31.367954 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.367933 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:28:31.368268 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.368253 2577 scope.go:117] "RemoveContainer" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" Jul 09 17:28:31.368454 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:31.368430 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rq8nf_openshift-console-operator(341cad21-7443-4c36-9c1e-29779538a458)\"" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podUID="341cad21-7443-4c36-9c1e-29779538a458" Jul 09 17:28:31.388605 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.388549 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bfldg" podStartSLOduration=17.597871208 podStartE2EDuration="53.388528096s" totalImagesPullingTime="35.790656888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:31.386327302 +0000 UTC m=+53.835379703" watchObservedRunningTime="2026-07-09 17:28:31.388528096 +0000 UTC m=+53.837580499" Jul 09 17:28:31.542637 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.542596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:31.542800 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:31.542720 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:28:31.542800 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:31.542736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:31.542800 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:31.542781 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:33.542765582 +0000 UTC m=+55.991817961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : configmap "kube-rbac-proxy" not found Jul 09 17:28:31.542922 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:31.542843 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:28:31.542922 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:31.542897 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:33.542882818 +0000 UTC m=+55.991935207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : secret "insights-runtime-extractor-tls" not found Jul 09 17:28:32.270410 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.270313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kjvx5_7d19f4e2-714d-458f-8d05-e5b2b92b03ab/node-ca/0.log" Jul 09 17:28:32.373501 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.373462 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" event={"ID":"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2","Type":"ContainerStarted","Data":"e98ab9552d233ec5bb7b170fd678ad7ce0b82dbdbe5470f3fc5b07a414d94458"} Jul 09 17:28:32.373930 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.373519 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" event={"ID":"7dc70c29-19e7-40f1-a41a-9bcc21f33bf2","Type":"ContainerStarted","Data":"1d622c27a9ea61fbee29307deaac66b602253bf46bfb4bcb6980d9cd9d78ae1b"} Jul 09 17:28:32.388405 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.388337 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-xn2m4" podStartSLOduration=1.931283941 podStartE2EDuration="3.388322654s" totalImagesPullingTime="1.457038713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:32.386469975 +0000 UTC m=+54.835522416" watchObservedRunningTime="2026-07-09 17:28:32.388322654 +0000 UTC m=+54.837375056" Jul 09 17:28:32.670046 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.670015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lbnts_41c280e1-4c69-4a64-895b-68aa2656f849/serve-healthcheck-canary/0.log" Jul 09 17:28:32.778853 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.778814 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:32.779023 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.778900 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:32.779201 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:32.779188 2577 scope.go:117] "RemoveContainer" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" Jul 09 17:28:32.779357 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:32.779338 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rq8nf_openshift-console-operator(341cad21-7443-4c36-9c1e-29779538a458)\"" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podUID="341cad21-7443-4c36-9c1e-29779538a458" Jul 09 17:28:33.075739 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.075711 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-f6sqx"] Jul 09 17:28:33.078885 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.078866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.081405 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.081360 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 17:28:33.081518 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.081437 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 17:28:33.082076 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.082060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6pt42\"" Jul 09 17:28:33.082159 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.082093 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 17:28:33.082215 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.082157 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 17:28:33.087870 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.087851 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-f6sqx"] Jul 09 17:28:33.158331 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.158291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7960140e-0b78-4db4-8d38-faac38673ec3-signing-cabundle\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.158533 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.158344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftk2j\" (UniqueName: \"kubernetes.io/projected/7960140e-0b78-4db4-8d38-faac38673ec3-kube-api-access-ftk2j\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.158533 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.158397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7960140e-0b78-4db4-8d38-faac38673ec3-signing-key\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.259059 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.259023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7960140e-0b78-4db4-8d38-faac38673ec3-signing-cabundle\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.259238 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.259080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftk2j\" (UniqueName: \"kubernetes.io/projected/7960140e-0b78-4db4-8d38-faac38673ec3-kube-api-access-ftk2j\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.259238 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.259115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7960140e-0b78-4db4-8d38-faac38673ec3-signing-key\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.259816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.259790 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7960140e-0b78-4db4-8d38-faac38673ec3-signing-cabundle\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.261581 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.261556 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7960140e-0b78-4db4-8d38-faac38673ec3-signing-key\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.267291 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.267268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftk2j\" (UniqueName: \"kubernetes.io/projected/7960140e-0b78-4db4-8d38-faac38673ec3-kube-api-access-ftk2j\") pod \"service-ca-79f68b9f4f-f6sqx\" (UID: \"7960140e-0b78-4db4-8d38-faac38673ec3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.377778 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.377705 2577 scope.go:117] "RemoveContainer" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" Jul 09 17:28:33.378121 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:33.377894 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rq8nf_openshift-console-operator(341cad21-7443-4c36-9c1e-29779538a458)\"" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podUID="341cad21-7443-4c36-9c1e-29779538a458" Jul 09 17:28:33.388568 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.388542 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" Jul 09 17:28:33.529760 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.529707 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-f6sqx"] Jul 09 17:28:33.532662 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:33.532634 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7960140e_0b78_4db4_8d38_faac38673ec3.slice/crio-cb923389d6f25c2f3eec2e709f3f0a3870b3a2f15abacb708b852f52855dcedb WatchSource:0}: Error finding container cb923389d6f25c2f3eec2e709f3f0a3870b3a2f15abacb708b852f52855dcedb: Status 404 returned error can't find the container with id cb923389d6f25c2f3eec2e709f3f0a3870b3a2f15abacb708b852f52855dcedb Jul 09 17:28:33.561617 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.561576 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:33.561801 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:33.561663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:33.561801 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:33.561739 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:28:33.561801 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:33.561781 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:28:33.561801 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:33.561798 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:37.561784262 +0000 UTC m=+60.010836642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : secret "insights-runtime-extractor-tls" not found Jul 09 17:28:33.561957 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:33.561829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:37.561817218 +0000 UTC m=+60.010869597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : configmap "kube-rbac-proxy" not found Jul 09 17:28:34.381935 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:34.381902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" event={"ID":"7960140e-0b78-4db4-8d38-faac38673ec3","Type":"ContainerStarted","Data":"47d13ed54fef4fafe9ea20254fe496bc1b88a768015c8d3705911c428a225c11"} Jul 09 17:28:34.382438 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:34.381949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" event={"ID":"7960140e-0b78-4db4-8d38-faac38673ec3","Type":"ContainerStarted","Data":"cb923389d6f25c2f3eec2e709f3f0a3870b3a2f15abacb708b852f52855dcedb"} Jul 09 17:28:34.399954 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:34.399904 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-f6sqx" podStartSLOduration=1.3998893940000001 podStartE2EDuration="1.399889394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:34.398690792 +0000 UTC m=+56.847743219" watchObservedRunningTime="2026-07-09 17:28:34.399889394 +0000 UTC m=+56.848941796" Jul 09 17:28:37.601020 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:37.600981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:37.601464 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:37.601063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:37.601464 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:37.601175 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:28:37.601464 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:37.601180 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:28:37.601464 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:37.601234 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:45.601218328 +0000 UTC m=+68.050270716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : configmap "kube-rbac-proxy" not found Jul 09 17:28:37.601464 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:37.601248 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls podName:9f6f190c-140b-4e27-9d61-d765bd2dfd4f nodeName:}" failed. No retries permitted until 2026-07-09 17:28:45.601242364 +0000 UTC m=+68.050294744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjkw7" (UID: "9f6f190c-140b-4e27-9d61-d765bd2dfd4f") : secret "insights-runtime-extractor-tls" not found Jul 09 17:28:38.107562 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:38.107520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:38.107710 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:38.107655 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:28:38.107773 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:28:38.107711 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert podName:4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa nodeName:}" failed. No retries permitted until 2026-07-09 17:28:54.107696496 +0000 UTC m=+76.556748882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jjwkq" (UID: "4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa") : secret "networking-console-plugin-cert" not found Jul 09 17:28:38.212177 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:38.212148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5q49f" Jul 09 17:28:42.853607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.853556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:42.854132 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.853625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:42.856545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.856526 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:28:42.857143 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.857130 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:28:42.867046 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.867021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/21fe8f0d-338f-4886-bed2-a7ec244552ba-original-pull-secret\") pod \"global-pull-secret-syncer-2dbb5\" (UID: \"21fe8f0d-338f-4886-bed2-a7ec244552ba\") " pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:42.867171 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.867096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431e5c77-b3ad-49b7-87d9-95ee5648429b-metrics-certs\") pod \"network-metrics-daemon-t5g2n\" (UID: \"431e5c77-b3ad-49b7-87d9-95ee5648429b\") " pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:42.908884 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.908854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ls9tf\"" Jul 09 17:28:42.912747 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.912724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t5g2n" Jul 09 17:28:42.912882 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.912844 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2dbb5" Jul 09 17:28:42.954786 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.954750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:42.957339 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:42.957307 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvhmk\" (UniqueName: \"kubernetes.io/projected/e4ad64d2-3ae5-4d0f-967b-b2856971662f-kube-api-access-cvhmk\") pod \"network-check-target-bwx97\" (UID: \"e4ad64d2-3ae5-4d0f-967b-b2856971662f\") " pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:43.110461 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.110431 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t5g2n"] Jul 09 17:28:43.117025 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:43.116992 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431e5c77_b3ad_49b7_87d9_95ee5648429b.slice/crio-f976905a9f7bd690234fc4f7b2707631faa78bff306ee62bc8e280fc61ea983e WatchSource:0}: Error finding container f976905a9f7bd690234fc4f7b2707631faa78bff306ee62bc8e280fc61ea983e: Status 404 returned error can't find the container with id f976905a9f7bd690234fc4f7b2707631faa78bff306ee62bc8e280fc61ea983e Jul 09 17:28:43.153985 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.153960 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2dbb5"] Jul 09 17:28:43.155316 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:43.155290 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21fe8f0d_338f_4886_bed2_a7ec244552ba.slice/crio-923c2ddfad8b2f07234de468925aa955b1b588b90a4f3a33af26904c0e8472c4 WatchSource:0}: Error finding container 923c2ddfad8b2f07234de468925aa955b1b588b90a4f3a33af26904c0e8472c4: Status 404 returned error can't find the container with id 923c2ddfad8b2f07234de468925aa955b1b588b90a4f3a33af26904c0e8472c4 Jul 09 17:28:43.193663 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.193627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8889q\"" Jul 09 17:28:43.202311 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.202287 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:43.369592 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.369510 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bwx97"] Jul 09 17:28:43.372384 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:43.372339 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ad64d2_3ae5_4d0f_967b_b2856971662f.slice/crio-9bc646953b42b75c97a2ddcf47873696efc42baece96ccbd1d13df0740bd198d WatchSource:0}: Error finding container 9bc646953b42b75c97a2ddcf47873696efc42baece96ccbd1d13df0740bd198d: Status 404 returned error can't find the container with id 9bc646953b42b75c97a2ddcf47873696efc42baece96ccbd1d13df0740bd198d Jul 09 17:28:43.415999 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.415964 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5g2n" event={"ID":"431e5c77-b3ad-49b7-87d9-95ee5648429b","Type":"ContainerStarted","Data":"f976905a9f7bd690234fc4f7b2707631faa78bff306ee62bc8e280fc61ea983e"} Jul 09 17:28:43.417233 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.417198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2dbb5" event={"ID":"21fe8f0d-338f-4886-bed2-a7ec244552ba","Type":"ContainerStarted","Data":"923c2ddfad8b2f07234de468925aa955b1b588b90a4f3a33af26904c0e8472c4"} Jul 09 17:28:43.418556 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:43.418526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bwx97" event={"ID":"e4ad64d2-3ae5-4d0f-967b-b2856971662f","Type":"ContainerStarted","Data":"9bc646953b42b75c97a2ddcf47873696efc42baece96ccbd1d13df0740bd198d"} Jul 09 17:28:44.373349 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.373303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:44.373816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.373359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:44.373816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.373423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:44.373816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.373503 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:44.374289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.374258 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/646b2559-7599-4b05-b2ab-fa37a13626fd-service-ca-bundle\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:44.376499 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.376456 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"image-registry-8575c6d767-6cxr9\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:44.376636 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.376554 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/646b2559-7599-4b05-b2ab-fa37a13626fd-metrics-certs\") pod \"router-default-54d66b6fb6-lfbzr\" (UID: \"646b2559-7599-4b05-b2ab-fa37a13626fd\") " pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:44.377397 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.377350 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10d58688-da25-40ec-9950-986e594fd257-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jfltc\" (UID: \"10d58688-da25-40ec-9950-986e594fd257\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:44.425394 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.425326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bwx97" event={"ID":"e4ad64d2-3ae5-4d0f-967b-b2856971662f","Type":"ContainerStarted","Data":"b5ef50a211c5ddc3bbe090aea3346fd84f64973f3f2c9398d3dad769fabeaf77"} Jul 09 17:28:44.425871 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.425622 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:28:44.441926 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.441863 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bwx97" podStartSLOduration=66.441842651 podStartE2EDuration="1m6.441842651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:44.441018947 +0000 UTC m=+66.890071370" watchObservedRunningTime="2026-07-09 17:28:44.441842651 +0000 UTC m=+66.890895056" Jul 09 17:28:44.474446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.474407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:44.477299 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.477258 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/291fddd5-d527-4a82-a144-e5aefb8aef5c-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-857c2\" (UID: \"291fddd5-d527-4a82-a144-e5aefb8aef5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:44.509414 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.509361 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cn56n\"" Jul 09 17:28:44.517150 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.517125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" Jul 09 17:28:44.540960 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.540922 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-m8drm\"" Jul 09 17:28:44.549902 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.549843 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:44.564674 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.564651 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6m8jk\"" Jul 09 17:28:44.573466 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.573440 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:44.575009 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.574985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:44.577864 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.577803 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/03c4b441-78be-49f8-8090-d0fadd5af852-metrics-tls\") pod \"dns-default-prsrd\" (UID: \"03c4b441-78be-49f8-8090-d0fadd5af852\") " pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:44.659909 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.659824 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4gsbp\"" Jul 09 17:28:44.668255 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.668226 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" Jul 09 17:28:44.695960 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.695933 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmgfn\"" Jul 09 17:28:44.704473 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.704418 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:44.855523 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.854268 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc"] Jul 09 17:28:44.859843 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:44.859099 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d58688_da25_40ec_9950_986e594fd257.slice/crio-c59d8f2496d38f9a75db2c54f581f26cc31caacbf26023963557fe3cec23f357 WatchSource:0}: Error finding container c59d8f2496d38f9a75db2c54f581f26cc31caacbf26023963557fe3cec23f357: Status 404 returned error can't find the container with id c59d8f2496d38f9a75db2c54f581f26cc31caacbf26023963557fe3cec23f357 Jul 09 17:28:44.914131 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.913974 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2"] Jul 09 17:28:44.942023 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:44.941999 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-prsrd"] Jul 09 17:28:45.041924 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:45.041881 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c4b441_78be_49f8_8090_d0fadd5af852.slice/crio-e3388c4fe0693631dfc22c6397e935ab615026f9489e5deb2d6ea7cc01b71104 WatchSource:0}: Error finding container e3388c4fe0693631dfc22c6397e935ab615026f9489e5deb2d6ea7cc01b71104: Status 404 returned error can't find the container with id e3388c4fe0693631dfc22c6397e935ab615026f9489e5deb2d6ea7cc01b71104 Jul 09 17:28:45.086849 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.086726 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54d66b6fb6-lfbzr"] Jul 09 17:28:45.090029 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.089965 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:28:45.090504 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:45.090407 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646b2559_7599_4b05_b2ab_fa37a13626fd.slice/crio-6d3314eb78aeb2a7bb0e1e87bc0738ee90001af89c7cdc6bb7577aa09c02b3b6 WatchSource:0}: Error finding container 6d3314eb78aeb2a7bb0e1e87bc0738ee90001af89c7cdc6bb7577aa09c02b3b6: Status 404 returned error can't find the container with id 6d3314eb78aeb2a7bb0e1e87bc0738ee90001af89c7cdc6bb7577aa09c02b3b6 Jul 09 17:28:45.093657 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:45.093629 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36d114c_d106_482d_9137_57aad81d6975.slice/crio-dcb80607b061797cd7ab86bb88bf806986e0c0b9c7e461d1237c287d8df1462b WatchSource:0}: Error finding container dcb80607b061797cd7ab86bb88bf806986e0c0b9c7e461d1237c287d8df1462b: Status 404 returned error can't find the container with id dcb80607b061797cd7ab86bb88bf806986e0c0b9c7e461d1237c287d8df1462b Jul 09 17:28:45.435769 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.435647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prsrd" event={"ID":"03c4b441-78be-49f8-8090-d0fadd5af852","Type":"ContainerStarted","Data":"e3388c4fe0693631dfc22c6397e935ab615026f9489e5deb2d6ea7cc01b71104"} Jul 09 17:28:45.437560 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.437525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" event={"ID":"f36d114c-d106-482d-9137-57aad81d6975","Type":"ContainerStarted","Data":"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7"} Jul 09 17:28:45.437711 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.437571 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" event={"ID":"f36d114c-d106-482d-9137-57aad81d6975","Type":"ContainerStarted","Data":"dcb80607b061797cd7ab86bb88bf806986e0c0b9c7e461d1237c287d8df1462b"} Jul 09 17:28:45.437711 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.437651 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:28:45.440486 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.440089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5g2n" event={"ID":"431e5c77-b3ad-49b7-87d9-95ee5648429b","Type":"ContainerStarted","Data":"0fa960b8baa29a9df6fd6585f498dd7d587010ff3c4d21a46a335a7f0815b8d5"} Jul 09 17:28:45.440486 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.440139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t5g2n" event={"ID":"431e5c77-b3ad-49b7-87d9-95ee5648429b","Type":"ContainerStarted","Data":"a2b85659d45142c836592f15d789ddfbb70301c2dbde53553df3c2739f6ffd42"} Jul 09 17:28:45.441744 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.441716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" event={"ID":"291fddd5-d527-4a82-a144-e5aefb8aef5c","Type":"ContainerStarted","Data":"16f8050aed22dc7dd1554be03d4ce16b122e4b61c76ad66db1dddab82d56f950"} Jul 09 17:28:45.443126 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.443102 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" event={"ID":"10d58688-da25-40ec-9950-986e594fd257","Type":"ContainerStarted","Data":"c59d8f2496d38f9a75db2c54f581f26cc31caacbf26023963557fe3cec23f357"} Jul 09 17:28:45.444927 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.444896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" event={"ID":"646b2559-7599-4b05-b2ab-fa37a13626fd","Type":"ContainerStarted","Data":"88867c5cbfed4833183fafb4267c49d0afe66ec53205356827a349e078b38525"} Jul 09 17:28:45.445041 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.444949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" event={"ID":"646b2559-7599-4b05-b2ab-fa37a13626fd","Type":"ContainerStarted","Data":"6d3314eb78aeb2a7bb0e1e87bc0738ee90001af89c7cdc6bb7577aa09c02b3b6"} Jul 09 17:28:45.456316 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.456267 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" podStartSLOduration=67.456247847 podStartE2EDuration="1m7.456247847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:45.455343357 +0000 UTC m=+67.904395785" watchObservedRunningTime="2026-07-09 17:28:45.456247847 +0000 UTC m=+67.905300251" Jul 09 17:28:45.473265 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.473215 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" podStartSLOduration=56.473201072 podStartE2EDuration="56.473201072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:45.471741384 +0000 UTC m=+67.920793807" watchObservedRunningTime="2026-07-09 17:28:45.473201072 +0000 UTC m=+67.922253472" Jul 09 17:28:45.487119 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.487068 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t5g2n" podStartSLOduration=65.537412003 podStartE2EDuration="1m7.487052479s" totalImagesPullingTime="1.949640476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:45.485604737 +0000 UTC m=+67.934657166" watchObservedRunningTime="2026-07-09 17:28:45.487052479 +0000 UTC m=+67.936104873" Jul 09 17:28:45.550427 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.550392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:45.553647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.553613 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:45.686523 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.686428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:45.686697 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.686520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:45.687329 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.687261 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:45.689343 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.689316 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f6f190c-140b-4e27-9d61-d765bd2dfd4f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjkw7\" (UID: \"9f6f190c-140b-4e27-9d61-d765bd2dfd4f\") " pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:45.794555 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.794512 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gxngq\"" Jul 09 17:28:45.802394 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.802347 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjkw7" Jul 09 17:28:45.985653 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:45.985469 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjkw7"] Jul 09 17:28:46.453259 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:46.452639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjkw7" event={"ID":"9f6f190c-140b-4e27-9d61-d765bd2dfd4f","Type":"ContainerStarted","Data":"b844610ba74ed2331dbc8e093bdaa2a7c77ce907acea9311591742aa7dab6fd9"} Jul 09 17:28:46.453259 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:46.453104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:46.454615 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:46.454424 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54d66b6fb6-lfbzr" Jul 09 17:28:47.069267 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:47.069199 2577 scope.go:117] "RemoveContainer" containerID="b9785a60ba3058f4d76e7e12f6b06985d4cf3c7f4cf3176b1714db31035d695d" Jul 09 17:28:50.481586 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.480601 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:28:50.481586 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.480698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rq8nf" event={"ID":"341cad21-7443-4c36-9c1e-29779538a458","Type":"ContainerStarted","Data":"3e7f77766ebce24fecfb1bd80576deae944a9730fcfe3b878aefd7fa093182d3"} Jul 09 17:28:50.481586 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.481530 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:50.482754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.482456 2577 patch_prober.go:28] interesting pod/console-operator-7866497494-rq8nf container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Jul 09 17:28:50.482754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.482507 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podUID="341cad21-7443-4c36-9c1e-29779538a458" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" Jul 09 17:28:50.501567 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:50.501513 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-rq8nf" podStartSLOduration=46.363286063 podStartE2EDuration="1m1.50149261s" totalImagesPullingTime="15.138206547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:50.499456038 +0000 UTC m=+72.948508472" watchObservedRunningTime="2026-07-09 17:28:50.50149261 +0000 UTC m=+72.950545013" Jul 09 17:28:51.488060 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.488016 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2dbb5" event={"ID":"21fe8f0d-338f-4886-bed2-a7ec244552ba","Type":"ContainerStarted","Data":"be565debfa4b6fa1597ad278f6bb76eccb9cfc0b38ba2ea04c81d40cd9249bc3"} Jul 09 17:28:51.489965 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.489934 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" event={"ID":"10d58688-da25-40ec-9950-986e594fd257","Type":"ContainerStarted","Data":"a6d85788d9ba64c5a36c7a9fc26c00a0d280840cf9a0898adeabc02b251023e5"} Jul 09 17:28:51.492016 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.491983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prsrd" event={"ID":"03c4b441-78be-49f8-8090-d0fadd5af852","Type":"ContainerStarted","Data":"968410234b7431e7628ffb8b80e6e15e4b0913d6532087f246ee6cd784f169b7"} Jul 09 17:28:51.492156 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.492030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prsrd" event={"ID":"03c4b441-78be-49f8-8090-d0fadd5af852","Type":"ContainerStarted","Data":"09876cd6b815f42215b8b61e7d52a431fa348fb33c72f155e694e2ee3e2f1360"} Jul 09 17:28:51.492156 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.492087 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-prsrd" Jul 09 17:28:51.494009 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.493983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" event={"ID":"291fddd5-d527-4a82-a144-e5aefb8aef5c","Type":"ContainerStarted","Data":"b997550f064be5c72aefc9f57427b7fca69c8e7c4261a8fb20f06217d489cce3"} Jul 09 17:28:51.494131 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.494033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" event={"ID":"291fddd5-d527-4a82-a144-e5aefb8aef5c","Type":"ContainerStarted","Data":"508ea45015329c4d7c0bc9b525b68f64d588f2bbe3c1fa40c7beb30077fecc2a"} Jul 09 17:28:51.499054 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.499030 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-rq8nf" Jul 09 17:28:51.503864 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.503799 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2dbb5" podStartSLOduration=66.300478663 podStartE2EDuration="1m13.503779284s" totalImagesPullingTime="7.203300621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:51.502238616 +0000 UTC m=+73.951291010" watchObservedRunningTime="2026-07-09 17:28:51.503779284 +0000 UTC m=+73.952831686" Jul 09 17:28:51.540662 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.540591 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jfltc" podStartSLOduration=57.218799646 podStartE2EDuration="1m2.540566284s" totalImagesPullingTime="5.321766638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:51.538702769 +0000 UTC m=+73.987755201" watchObservedRunningTime="2026-07-09 17:28:51.540566284 +0000 UTC m=+73.989618697" Jul 09 17:28:51.555418 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.555349 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-prsrd" podStartSLOduration=34.258413871 podStartE2EDuration="39.555331225s" totalImagesPullingTime="5.296917354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:51.553752836 +0000 UTC m=+74.002805263" watchObservedRunningTime="2026-07-09 17:28:51.555331225 +0000 UTC m=+74.004383627" Jul 09 17:28:51.570656 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:51.570493 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-857c2" podStartSLOduration=57.345059344 podStartE2EDuration="1m2.570474347s" totalImagesPullingTime="5.225415003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:51.570013072 +0000 UTC m=+74.019065505" watchObservedRunningTime="2026-07-09 17:28:51.570474347 +0000 UTC m=+74.019526750" Jul 09 17:28:54.167897 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.167860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:54.170313 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.170286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jjwkq\" (UID: \"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:54.294454 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.294424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5q8ns\"" Jul 09 17:28:54.303428 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.303404 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" Jul 09 17:28:54.467957 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.467887 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jjwkq"] Jul 09 17:28:54.509012 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:54.508970 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjkw7" event={"ID":"9f6f190c-140b-4e27-9d61-d765bd2dfd4f","Type":"ContainerStarted","Data":"f79ae0a28c8f23244dd8b0b91a75eab5ba054619526a4132df428f9694c033ad"} Jul 09 17:28:54.541023 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:28:54.540985 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd31e0c_76d6_4d77_8f6e_a9ef47b73bfa.slice/crio-870c8111ddc2ac6c470457f7cbf0bfb0b70999251597aa92faef9cf4957e94a4 WatchSource:0}: Error finding container 870c8111ddc2ac6c470457f7cbf0bfb0b70999251597aa92faef9cf4957e94a4: Status 404 returned error can't find the container with id 870c8111ddc2ac6c470457f7cbf0bfb0b70999251597aa92faef9cf4957e94a4 Jul 09 17:28:55.515259 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:55.515220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjkw7" event={"ID":"9f6f190c-140b-4e27-9d61-d765bd2dfd4f","Type":"ContainerStarted","Data":"3c37c903fc776d52539cf916afd06971e4d7ffbd477f885b62a11e0b5bca75f5"} Jul 09 17:28:55.516712 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:55.516684 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" event={"ID":"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa","Type":"ContainerStarted","Data":"870c8111ddc2ac6c470457f7cbf0bfb0b70999251597aa92faef9cf4957e94a4"} Jul 09 17:28:57.525252 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:57.525213 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjkw7" event={"ID":"9f6f190c-140b-4e27-9d61-d765bd2dfd4f","Type":"ContainerStarted","Data":"c7d72bc63f51b9cad52dd2525c637167c6285f30e90712fe323d8e0409ca505e"} Jul 09 17:28:57.527186 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:57.527161 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" event={"ID":"4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa","Type":"ContainerStarted","Data":"a870bfb169a0e34494a77bdafcefdc7015697a0afd5f158d350c2ba32801f026"} Jul 09 17:28:57.542964 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:57.542867 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fjkw7" podStartSLOduration=17.344887793 podStartE2EDuration="28.542850892s" totalImagesPullingTime="11.197963099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:28:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:57.541674008 +0000 UTC m=+79.990726434" watchObservedRunningTime="2026-07-09 17:28:57.542850892 +0000 UTC m=+79.991903292" Jul 09 17:28:57.556123 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:28:57.556074 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-jjwkq" podStartSLOduration=32.803561318 podStartE2EDuration="35.556058154s" totalImagesPullingTime="2.752496836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:28:57.554984828 +0000 UTC m=+80.004037250" watchObservedRunningTime="2026-07-09 17:28:57.556058154 +0000 UTC m=+80.005110555" Jul 09 17:29:01.501841 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:01.501710 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-prsrd" Jul 09 17:29:04.578186 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:04.578151 2577 patch_prober.go:28] interesting pod/image-registry-8575c6d767-6cxr9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 17:29:04.578676 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:04.578220 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" podUID="f36d114c-d106-482d-9137-57aad81d6975" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 17:29:06.458713 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:06.458419 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:29:08.479283 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.479246 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gvmn6"] Jul 09 17:29:08.485128 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.485104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.487633 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.487612 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 17:29:08.487997 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.487974 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mtzcm\"" Jul 09 17:29:08.488109 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.487975 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 17:29:08.488520 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.488501 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 17:29:08.488620 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.488541 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 17:29:08.584967 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.584929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.584985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb8gt\" (UniqueName: \"kubernetes.io/projected/92b304e3-18a1-4981-9a25-74d3c9df4c2c-kube-api-access-hb8gt\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-metrics-client-ca\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-root\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-textfile\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-sys\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-wtmp\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.585441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.585302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.686600 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-root\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.686915 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686779 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-root\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.686915 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-textfile\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.686915 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-sys\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.687110 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.687110 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686961 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-wtmp\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.687251 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-sys\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.686997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.687460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.687500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb8gt\" (UniqueName: \"kubernetes.io/projected/92b304e3-18a1-4981-9a25-74d3c9df4c2c-kube-api-access-hb8gt\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.687503 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-textfile\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.687550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-metrics-client-ca\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.687213 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-wtmp\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:29:08.687091 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:29:08.687870 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls podName:92b304e3-18a1-4981-9a25-74d3c9df4c2c nodeName:}" failed. No retries permitted until 2026-07-09 17:29:09.187850225 +0000 UTC m=+91.636902608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls") pod "node-exporter-gvmn6" (UID: "92b304e3-18a1-4981-9a25-74d3c9df4c2c") : secret "node-exporter-tls" not found Jul 09 17:29:08.688087 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.688088 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-metrics-client-ca\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.688552 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.688131 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-accelerators-collector-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.690199 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.690142 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:08.708445 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:08.708353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb8gt\" (UniqueName: \"kubernetes.io/projected/92b304e3-18a1-4981-9a25-74d3c9df4c2c-kube-api-access-hb8gt\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:09.192890 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:09.192848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:09.195318 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:09.195290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92b304e3-18a1-4981-9a25-74d3c9df4c2c-node-exporter-tls\") pod \"node-exporter-gvmn6\" (UID: \"92b304e3-18a1-4981-9a25-74d3c9df4c2c\") " pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:09.396156 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:09.396123 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gvmn6" Jul 09 17:29:09.413846 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:29:09.413812 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b304e3_18a1_4981_9a25_74d3c9df4c2c.slice/crio-9a0ad136312f4738c57cf8013af6a1e138ef8f2c460e57b43bb59bf172be6553 WatchSource:0}: Error finding container 9a0ad136312f4738c57cf8013af6a1e138ef8f2c460e57b43bb59bf172be6553: Status 404 returned error can't find the container with id 9a0ad136312f4738c57cf8013af6a1e138ef8f2c460e57b43bb59bf172be6553 Jul 09 17:29:09.576639 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:09.576605 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gvmn6" event={"ID":"92b304e3-18a1-4981-9a25-74d3c9df4c2c","Type":"ContainerStarted","Data":"9a0ad136312f4738c57cf8013af6a1e138ef8f2c460e57b43bb59bf172be6553"} Jul 09 17:29:11.590949 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:11.590909 2577 generic.go:356] "Generic (PLEG): container finished" podID="92b304e3-18a1-4981-9a25-74d3c9df4c2c" containerID="5a5d6acc253662f7ff30e215fad40bb53fefcd959cade6c20bdd6061ee27d015" exitCode=0 Jul 09 17:29:11.591317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:11.590991 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gvmn6" event={"ID":"92b304e3-18a1-4981-9a25-74d3c9df4c2c","Type":"ContainerDied","Data":"5a5d6acc253662f7ff30e215fad40bb53fefcd959cade6c20bdd6061ee27d015"} Jul 09 17:29:12.596765 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:12.596729 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gvmn6" event={"ID":"92b304e3-18a1-4981-9a25-74d3c9df4c2c","Type":"ContainerStarted","Data":"7062aa608e130c5341c3e70003c84cb447b8a93eaf8b21cb0f3c6cd31107d155"} Jul 09 17:29:12.597233 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:12.596790 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gvmn6" event={"ID":"92b304e3-18a1-4981-9a25-74d3c9df4c2c","Type":"ContainerStarted","Data":"41bd53e834ed1b59e5567febf87ff547c141e847673326b96771d94eac4403d8"} Jul 09 17:29:12.616573 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:12.616525 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gvmn6" podStartSLOduration=3.428050104 podStartE2EDuration="4.616509703s" totalImagesPullingTime="1.12385959s" totalInitContainerRuntime="64.600009ms" isStatefulPod=true podCreationTimestamp="2026-07-09 17:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:29:12.615552388 +0000 UTC m=+95.064604788" watchObservedRunningTime="2026-07-09 17:29:12.616509703 +0000 UTC m=+95.065562105" Jul 09 17:29:15.447793 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:15.447762 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bwx97" Jul 09 17:29:15.948292 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:15.948255 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:29:39.701931 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:39.701898 2577 generic.go:356] "Generic (PLEG): container finished" podID="d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b" containerID="65ec7610f67325bcfbd16acaff898c3e45fef24622433d9dc16386ec2bf450d5" exitCode=0 Jul 09 17:29:39.702347 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:39.701971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" event={"ID":"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b","Type":"ContainerDied","Data":"65ec7610f67325bcfbd16acaff898c3e45fef24622433d9dc16386ec2bf450d5"} Jul 09 17:29:39.702495 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:39.702386 2577 scope.go:117] "RemoveContainer" containerID="65ec7610f67325bcfbd16acaff898c3e45fef24622433d9dc16386ec2bf450d5" Jul 09 17:29:40.708589 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:40.708555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-mwdxg" event={"ID":"d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b","Type":"ContainerStarted","Data":"85b7138aa76831bfc8b9572f57b02c52df69155aa37fa7583f7557b19462992d"} Jul 09 17:29:40.969295 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:40.969200 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" podUID="f36d114c-d106-482d-9137-57aad81d6975" containerName="registry" containerID="cri-o://842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7" gracePeriod=30 Jul 09 17:29:41.219896 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.219834 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:29:41.368482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368453 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368492 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368528 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368548 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368570 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpbbp\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368595 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368651 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.368709 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.368677 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca\") pod \"f36d114c-d106-482d-9137-57aad81d6975\" (UID: \"f36d114c-d106-482d-9137-57aad81d6975\") " Jul 09 17:29:41.369545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.369104 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:29:41.369545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.369216 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:29:41.371177 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.371148 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:29:41.371290 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.371178 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:29:41.371290 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.371255 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp" (OuterVolumeSpecName: "kube-api-access-fpbbp") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "kube-api-access-fpbbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:29:41.371290 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.371268 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:29:41.371427 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.371404 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:29:41.377512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.377480 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f36d114c-d106-482d-9137-57aad81d6975" (UID: "f36d114c-d106-482d-9137-57aad81d6975"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:29:41.469865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469827 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-registry-certificates\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.469865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469861 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-bound-sa-token\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.469865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469871 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-installation-pull-secrets\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.470097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469880 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpbbp\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-kube-api-access-fpbbp\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.470097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469888 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36d114c-d106-482d-9137-57aad81d6975-ca-trust-extracted\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.470097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469897 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36d114c-d106-482d-9137-57aad81d6975-registry-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.470097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469905 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36d114c-d106-482d-9137-57aad81d6975-trusted-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.470097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.469914 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36d114c-d106-482d-9137-57aad81d6975-image-registry-private-configuration\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:29:41.713351 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.713306 2577 generic.go:356] "Generic (PLEG): container finished" podID="f36d114c-d106-482d-9137-57aad81d6975" containerID="842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7" exitCode=0 Jul 09 17:29:41.713351 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.713355 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" Jul 09 17:29:41.713792 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.713397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" event={"ID":"f36d114c-d106-482d-9137-57aad81d6975","Type":"ContainerDied","Data":"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7"} Jul 09 17:29:41.713792 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.713444 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8575c6d767-6cxr9" event={"ID":"f36d114c-d106-482d-9137-57aad81d6975","Type":"ContainerDied","Data":"dcb80607b061797cd7ab86bb88bf806986e0c0b9c7e461d1237c287d8df1462b"} Jul 09 17:29:41.713792 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.713468 2577 scope.go:117] "RemoveContainer" containerID="842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7" Jul 09 17:29:41.722037 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.722020 2577 scope.go:117] "RemoveContainer" containerID="842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7" Jul 09 17:29:41.722322 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:29:41.722302 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7\": container with ID starting with 842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7 not found: ID does not exist" containerID="842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7" Jul 09 17:29:41.722382 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.722331 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7"} err="failed to get container status \"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7\": rpc error: code = NotFound desc = could not find container \"842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7\": container with ID starting with 842a12b876be20135a984c09867393074d26092e29b6200f68d8896b045451a7 not found: ID does not exist" Jul 09 17:29:41.733706 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.733681 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:29:41.737607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:41.737585 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8575c6d767-6cxr9"] Jul 09 17:29:42.073894 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:42.073866 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36d114c-d106-482d-9137-57aad81d6975" path="/var/lib/kubelet/pods/f36d114c-d106-482d-9137-57aad81d6975/volumes" Jul 09 17:29:43.001385 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:43.001318 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" podUID="77588946-08cd-4216-8206-6b718aa3e4e2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 17:29:49.744633 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:49.744544 2577 generic.go:356] "Generic (PLEG): container finished" podID="bce3654b-14f1-431b-98a6-d82060ce7332" containerID="7c97ed55246bd11c096233b39bf4c41c4e648c6bab536bdb5260ab01bab6bde5" exitCode=0 Jul 09 17:29:49.744633 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:49.744619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" event={"ID":"bce3654b-14f1-431b-98a6-d82060ce7332","Type":"ContainerDied","Data":"7c97ed55246bd11c096233b39bf4c41c4e648c6bab536bdb5260ab01bab6bde5"} Jul 09 17:29:49.745046 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:49.744973 2577 scope.go:117] "RemoveContainer" containerID="7c97ed55246bd11c096233b39bf4c41c4e648c6bab536bdb5260ab01bab6bde5" Jul 09 17:29:50.751086 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:50.751049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7jb9q" event={"ID":"bce3654b-14f1-431b-98a6-d82060ce7332","Type":"ContainerStarted","Data":"4ffd0ecf0b9307c86d017c9d9b6c4fb5d65f7a196cd83af2ebbfddbb1ece619a"} Jul 09 17:29:53.001576 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:53.001537 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" podUID="77588946-08cd-4216-8206-6b718aa3e4e2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 17:29:59.787339 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:59.787254 2577 generic.go:356] "Generic (PLEG): container finished" podID="bedd31c3-14f3-42f6-8da8-133b13e061e3" containerID="2290a7aad68b4db250b76c585cac57050d19e68546cb2015d0d20bc003dfa70c" exitCode=0 Jul 09 17:29:59.787801 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:59.787336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" event={"ID":"bedd31c3-14f3-42f6-8da8-133b13e061e3","Type":"ContainerDied","Data":"2290a7aad68b4db250b76c585cac57050d19e68546cb2015d0d20bc003dfa70c"} Jul 09 17:29:59.787801 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:29:59.787794 2577 scope.go:117] "RemoveContainer" containerID="2290a7aad68b4db250b76c585cac57050d19e68546cb2015d0d20bc003dfa70c" Jul 09 17:30:00.793246 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:00.793216 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-66x4d" event={"ID":"bedd31c3-14f3-42f6-8da8-133b13e061e3","Type":"ContainerStarted","Data":"0b27b487683caf17540450be4915922bbae2887939e7b6199bd6acb9ea326beb"} Jul 09 17:30:03.000862 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.000821 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" podUID="77588946-08cd-4216-8206-6b718aa3e4e2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 17:30:03.001223 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.000894 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" Jul 09 17:30:03.001437 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.001400 2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3fc0c0c88ddf008c2d3a1ca237a110dd2c3e04b2ffa24daf6531aae13f8a19f5"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 09 17:30:03.001483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.001457 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" podUID="77588946-08cd-4216-8206-6b718aa3e4e2" containerName="service-proxy" containerID="cri-o://3fc0c0c88ddf008c2d3a1ca237a110dd2c3e04b2ffa24daf6531aae13f8a19f5" gracePeriod=30 Jul 09 17:30:03.806312 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.806270 2577 generic.go:356] "Generic (PLEG): container finished" podID="77588946-08cd-4216-8206-6b718aa3e4e2" containerID="3fc0c0c88ddf008c2d3a1ca237a110dd2c3e04b2ffa24daf6531aae13f8a19f5" exitCode=2 Jul 09 17:30:03.806512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.806327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerDied","Data":"3fc0c0c88ddf008c2d3a1ca237a110dd2c3e04b2ffa24daf6531aae13f8a19f5"} Jul 09 17:30:03.806512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:30:03.806361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c4779f69-rt96c" event={"ID":"77588946-08cd-4216-8206-6b718aa3e4e2","Type":"ContainerStarted","Data":"a001dbd021876abd5f681d91e50e38195eb5761673c093ccf1e48ba6ab440d83"} Jul 09 17:32:25.589943 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.589910 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7"] Jul 09 17:32:25.590393 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.590249 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36d114c-d106-482d-9137-57aad81d6975" containerName="registry" Jul 09 17:32:25.590393 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.590260 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36d114c-d106-482d-9137-57aad81d6975" containerName="registry" Jul 09 17:32:25.590393 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.590314 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36d114c-d106-482d-9137-57aad81d6975" containerName="registry" Jul 09 17:32:25.593326 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.593302 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.595932 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.595902 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 17:32:25.596098 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.596071 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 17:32:25.596223 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.596202 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-94t5d\"" Jul 09 17:32:25.596283 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.596203 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 17:32:25.596737 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.596720 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 17:32:25.616117 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.616086 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7"] Jul 09 17:32:25.687684 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.687640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8gl8\" (UniqueName: \"kubernetes.io/projected/c8b91e09-c552-43a5-b56f-1048f059c5f0-kube-api-access-r8gl8\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.687880 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.687733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.687880 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.687760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.687880 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.687826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-manifests\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.687880 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.687862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-tmp\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.788743 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.788702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.788743 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.788744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.789001 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.788781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-manifests\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.789001 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.788800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-tmp\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.789001 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.788834 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8gl8\" (UniqueName: \"kubernetes.io/projected/c8b91e09-c552-43a5-b56f-1048f059c5f0-kube-api-access-r8gl8\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.789284 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.789259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-manifests\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.791268 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.791234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b91e09-c552-43a5-b56f-1048f059c5f0-tmp\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.791441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.791400 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.791500 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.791467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b91e09-c552-43a5-b56f-1048f059c5f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.797646 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.797619 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8gl8\" (UniqueName: \"kubernetes.io/projected/c8b91e09-c552-43a5-b56f-1048f059c5f0-kube-api-access-r8gl8\") pod \"opendatahub-operator-controller-manager-c8444b686-2m6z7\" (UID: \"c8b91e09-c552-43a5-b56f-1048f059c5f0\") " pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:25.903919 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:25.903819 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:26.050720 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:26.050684 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7"] Jul 09 17:32:26.054432 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:32:26.054359 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b91e09_c552_43a5_b56f_1048f059c5f0.slice/crio-d26bd5998092895283fa1d5435cf00992a2e71463f90dc72bc20cb23e6570fba WatchSource:0}: Error finding container d26bd5998092895283fa1d5435cf00992a2e71463f90dc72bc20cb23e6570fba: Status 404 returned error can't find the container with id d26bd5998092895283fa1d5435cf00992a2e71463f90dc72bc20cb23e6570fba Jul 09 17:32:26.351837 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:26.351800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" event={"ID":"c8b91e09-c552-43a5-b56f-1048f059c5f0","Type":"ContainerStarted","Data":"d26bd5998092895283fa1d5435cf00992a2e71463f90dc72bc20cb23e6570fba"} Jul 09 17:32:29.368444 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:29.368407 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" event={"ID":"c8b91e09-c552-43a5-b56f-1048f059c5f0","Type":"ContainerStarted","Data":"b983d427a2594706fed52c26f0f52b3dce61c5309392db19c95c089396f2d074"} Jul 09 17:32:31.378408 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:31.378352 2577 generic.go:356] "Generic (PLEG): container finished" podID="c8b91e09-c552-43a5-b56f-1048f059c5f0" containerID="b983d427a2594706fed52c26f0f52b3dce61c5309392db19c95c089396f2d074" exitCode=0 Jul 09 17:32:31.378838 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:31.378425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" event={"ID":"c8b91e09-c552-43a5-b56f-1048f059c5f0","Type":"ContainerDied","Data":"b983d427a2594706fed52c26f0f52b3dce61c5309392db19c95c089396f2d074"} Jul 09 17:32:32.384460 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:32.384421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" event={"ID":"c8b91e09-c552-43a5-b56f-1048f059c5f0","Type":"ContainerStarted","Data":"c4acec7a8b60ed23f6c0fdc02698948656d4aa43cff62be1469cdb76fff44712"} Jul 09 17:32:32.384863 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:32.384642 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:32.405256 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:32.405202 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" podStartSLOduration=2.557984417 podStartE2EDuration="7.40518646s" totalImagesPullingTime="3.762415219s" totalInitContainerRuntime="1.084786824s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:32:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:32:32.403931922 +0000 UTC m=+294.852984373" watchObservedRunningTime="2026-07-09 17:32:32.40518646 +0000 UTC m=+294.854238861" Jul 09 17:32:37.977577 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:37.977542 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:32:37.980287 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:37.980262 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:32:37.989327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:37.989303 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 17:32:43.391223 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:43.391186 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c8444b686-2m6z7" Jul 09 17:32:55.882260 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.882226 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8"] Jul 09 17:32:55.885497 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.885479 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:55.890580 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.890552 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 17:32:55.890848 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.890829 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 17:32:55.897897 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.897879 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 17:32:55.898193 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.898181 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 17:32:55.899268 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.899245 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:32:55.899329 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.899245 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2d9k8\"" Jul 09 17:32:55.912592 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.912568 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8"] Jul 09 17:32:55.943382 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.943322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-manager-config\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:55.943545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.943398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:55.943545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.943439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxlfh\" (UniqueName: \"kubernetes.io/projected/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-kube-api-access-fxlfh\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:55.943545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:55.943514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-metrics-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.044497 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.044461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-metrics-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.044687 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.044525 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-manager-config\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.044687 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.044556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.044687 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.044584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxlfh\" (UniqueName: \"kubernetes.io/projected/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-kube-api-access-fxlfh\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.045338 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.045309 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-manager-config\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.047257 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.047224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.047402 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.047316 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-metrics-cert\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.081050 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.081014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxlfh\" (UniqueName: \"kubernetes.io/projected/3192ba7a-3fce-4396-8bc2-e1ad70dd960d-kube-api-access-fxlfh\") pod \"lws-controller-manager-6784c458c5-sjtw8\" (UID: \"3192ba7a-3fce-4396-8bc2-e1ad70dd960d\") " pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.192998 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.192916 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7"] Jul 09 17:32:56.195116 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.195088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:32:56.196555 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.196536 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.199173 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.199155 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 17:32:56.199268 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.199240 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fq84m\"" Jul 09 17:32:56.210048 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.210024 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7"] Jul 09 17:32:56.246452 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.246341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.246647 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.246502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25ldv\" (UniqueName: \"kubernetes.io/projected/c11272d8-2657-41bc-b218-c2565d8bc5a4-kube-api-access-25ldv\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.337235 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.337210 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8"] Jul 09 17:32:56.339688 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:32:56.339659 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3192ba7a_3fce_4396_8bc2_e1ad70dd960d.slice/crio-744e8141485abfe0465d759a0032ba40fbf8ebf4b20b2dd99d70265d22358438 WatchSource:0}: Error finding container 744e8141485abfe0465d759a0032ba40fbf8ebf4b20b2dd99d70265d22358438: Status 404 returned error can't find the container with id 744e8141485abfe0465d759a0032ba40fbf8ebf4b20b2dd99d70265d22358438 Jul 09 17:32:56.342245 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.342221 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:32:56.347246 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.347221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.347353 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.347297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ldv\" (UniqueName: \"kubernetes.io/projected/c11272d8-2657-41bc-b218-c2565d8bc5a4-kube-api-access-25ldv\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.347431 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:32:56.347408 2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 17:32:56.347511 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:32:56.347492 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert podName:c11272d8-2657-41bc-b218-c2565d8bc5a4 nodeName:}" failed. No retries permitted until 2026-07-09 17:32:56.847469203 +0000 UTC m=+319.296521591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert") pod "llmisvc-controller-manager-777c6bd7d5-btwq7" (UID: "c11272d8-2657-41bc-b218-c2565d8bc5a4") : secret "llmisvc-webhook-server-cert" not found Jul 09 17:32:56.369666 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.369631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ldv\" (UniqueName: \"kubernetes.io/projected/c11272d8-2657-41bc-b218-c2565d8bc5a4-kube-api-access-25ldv\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.492103 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.492019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" event={"ID":"3192ba7a-3fce-4396-8bc2-e1ad70dd960d","Type":"ContainerStarted","Data":"744e8141485abfe0465d759a0032ba40fbf8ebf4b20b2dd99d70265d22358438"} Jul 09 17:32:56.850451 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.850420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:56.852805 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:56.852774 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c11272d8-2657-41bc-b218-c2565d8bc5a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-btwq7\" (UID: \"c11272d8-2657-41bc-b218-c2565d8bc5a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:57.125955 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:57.125872 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:32:57.254783 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:57.254760 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7"] Jul 09 17:32:57.256950 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:32:57.256917 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc11272d8_2657_41bc_b218_c2565d8bc5a4.slice/crio-adcf6bca56448eef29aaf0783c0ed50b7945bf388f30e620f4384aae8f239200 WatchSource:0}: Error finding container adcf6bca56448eef29aaf0783c0ed50b7945bf388f30e620f4384aae8f239200: Status 404 returned error can't find the container with id adcf6bca56448eef29aaf0783c0ed50b7945bf388f30e620f4384aae8f239200 Jul 09 17:32:57.504561 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:32:57.499074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" event={"ID":"c11272d8-2657-41bc-b218-c2565d8bc5a4","Type":"ContainerStarted","Data":"adcf6bca56448eef29aaf0783c0ed50b7945bf388f30e620f4384aae8f239200"} Jul 09 17:33:02.525059 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.525017 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" event={"ID":"3192ba7a-3fce-4396-8bc2-e1ad70dd960d","Type":"ContainerStarted","Data":"c2709bc24da45cdb8cf5a65c2c5459025a0bd846427ba5f0ed472237fc1890c2"} Jul 09 17:33:02.525535 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.525099 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:33:02.526625 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.526602 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" event={"ID":"c11272d8-2657-41bc-b218-c2565d8bc5a4","Type":"ContainerStarted","Data":"301734105e22ddbb9a3a4957ffcac901fb8361bf51585c7d9531434a831f4e0f"} Jul 09 17:33:02.526775 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.526759 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:33:02.542082 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.542030 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" podStartSLOduration=2.076236889 podStartE2EDuration="7.542014338s" totalImagesPullingTime="5.465777449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:33:02.539840398 +0000 UTC m=+324.988892850" watchObservedRunningTime="2026-07-09 17:33:02.542014338 +0000 UTC m=+324.991066739" Jul 09 17:33:02.555078 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:02.555024 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" podStartSLOduration=2.00399529 podStartE2EDuration="6.555007225s" totalImagesPullingTime="4.551011935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:33:02.554496098 +0000 UTC m=+325.003548536" watchObservedRunningTime="2026-07-09 17:33:02.555007225 +0000 UTC m=+325.004059628" Jul 09 17:33:13.533716 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:13.533680 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6784c458c5-sjtw8" Jul 09 17:33:33.532553 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:33.532473 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-btwq7" Jul 09 17:33:47.166642 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.166602 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlhkv"] Jul 09 17:33:47.169891 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.169873 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:47.172110 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.172090 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 17:33:47.172951 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.172935 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xbfmb\"" Jul 09 17:33:47.173047 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.172967 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 17:33:47.177942 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.177922 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlhkv"] Jul 09 17:33:47.261015 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.260979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdfb5\" (UniqueName: \"kubernetes.io/projected/515e4ca0-4ab3-42b8-ba04-e74f4a367b3e-kube-api-access-xdfb5\") pod \"kuadrant-operator-catalog-jlhkv\" (UID: \"515e4ca0-4ab3-42b8-ba04-e74f4a367b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:47.362045 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.362009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdfb5\" (UniqueName: \"kubernetes.io/projected/515e4ca0-4ab3-42b8-ba04-e74f4a367b3e-kube-api-access-xdfb5\") pod \"kuadrant-operator-catalog-jlhkv\" (UID: \"515e4ca0-4ab3-42b8-ba04-e74f4a367b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:47.370009 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.369980 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdfb5\" (UniqueName: \"kubernetes.io/projected/515e4ca0-4ab3-42b8-ba04-e74f4a367b3e-kube-api-access-xdfb5\") pod \"kuadrant-operator-catalog-jlhkv\" (UID: \"515e4ca0-4ab3-42b8-ba04-e74f4a367b3e\") " pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:47.479450 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.479346 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:47.610667 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.610642 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlhkv"] Jul 09 17:33:47.614276 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:33:47.614251 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515e4ca0_4ab3_42b8_ba04_e74f4a367b3e.slice/crio-b56c24c356904b6caac2bf2c1e578c8a9e6a6d6a0cdddc668486184db6c23b63 WatchSource:0}: Error finding container b56c24c356904b6caac2bf2c1e578c8a9e6a6d6a0cdddc668486184db6c23b63: Status 404 returned error can't find the container with id b56c24c356904b6caac2bf2c1e578c8a9e6a6d6a0cdddc668486184db6c23b63 Jul 09 17:33:47.713792 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:47.713758 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" event={"ID":"515e4ca0-4ab3-42b8-ba04-e74f4a367b3e","Type":"ContainerStarted","Data":"b56c24c356904b6caac2bf2c1e578c8a9e6a6d6a0cdddc668486184db6c23b63"} Jul 09 17:33:49.726497 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:49.726446 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" event={"ID":"515e4ca0-4ab3-42b8-ba04-e74f4a367b3e","Type":"ContainerStarted","Data":"62fdb871927b7e019df3b1825ff2b12acae3ec14031cda482d92f4e1f6ce3476"} Jul 09 17:33:49.742310 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:49.742263 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" podStartSLOduration=0.844536657 podStartE2EDuration="2.742247694s" totalImagesPullingTime="1.897711037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:33:49.740824558 +0000 UTC m=+372.189876961" watchObservedRunningTime="2026-07-09 17:33:49.742247694 +0000 UTC m=+372.191300096" Jul 09 17:33:57.479761 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:57.479714 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:57.480141 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:57.479813 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:57.502003 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:57.501976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:33:57.781933 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:33:57.781862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jlhkv" Jul 09 17:34:26.601118 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.601087 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln"] Jul 09 17:34:26.609451 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.609429 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:26.614434 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.614411 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8579q\"" Jul 09 17:34:26.621288 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.621264 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln"] Jul 09 17:34:26.687130 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.687089 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff7xw\" (UniqueName: \"kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw\") pod \"limitador-operator-controller-manager-859d7b78bd-htqln\" (UID: \"72f001a7-94b3-42a5-9175-aa157be455bf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:26.788274 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.788242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff7xw\" (UniqueName: \"kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw\") pod \"limitador-operator-controller-manager-859d7b78bd-htqln\" (UID: \"72f001a7-94b3-42a5-9175-aa157be455bf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:26.796428 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.796395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff7xw\" (UniqueName: \"kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw\") pod \"limitador-operator-controller-manager-859d7b78bd-htqln\" (UID: \"72f001a7-94b3-42a5-9175-aa157be455bf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:26.919773 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:26.919687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:27.072278 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:27.072243 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln"] Jul 09 17:34:27.078123 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:34:27.078095 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f001a7_94b3_42a5_9175_aa157be455bf.slice/crio-8eb0debab1d9a0f435482a04f5169676ba742ed713891f3cc89d7762cde61bd0 WatchSource:0}: Error finding container 8eb0debab1d9a0f435482a04f5169676ba742ed713891f3cc89d7762cde61bd0: Status 404 returned error can't find the container with id 8eb0debab1d9a0f435482a04f5169676ba742ed713891f3cc89d7762cde61bd0 Jul 09 17:34:27.888742 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:27.888705 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" event={"ID":"72f001a7-94b3-42a5-9175-aa157be455bf","Type":"ContainerStarted","Data":"8eb0debab1d9a0f435482a04f5169676ba742ed713891f3cc89d7762cde61bd0"} Jul 09 17:34:29.900412 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:29.900360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" event={"ID":"72f001a7-94b3-42a5-9175-aa157be455bf","Type":"ContainerStarted","Data":"d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660"} Jul 09 17:34:29.900787 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:29.900497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:29.920335 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:29.920286 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" podStartSLOduration=2.015205646 podStartE2EDuration="3.920271877s" totalImagesPullingTime="1.905066231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:29.919483613 +0000 UTC m=+412.368536038" watchObservedRunningTime="2026-07-09 17:34:29.920271877 +0000 UTC m=+412.369324312" Jul 09 17:34:40.383149 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.383111 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln"] Jul 09 17:34:40.383610 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.383306 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" containerName="manager" containerID="cri-o://d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660" gracePeriod=2 Jul 09 17:34:40.385137 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.385112 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.396538 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.396513 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln"] Jul 09 17:34:40.403155 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.403123 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"72f001a7-94b3-42a5-9175-aa157be455bf\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-htqln\": pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.405243 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.405212 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.413603 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.413580 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v"] Jul 09 17:34:40.413910 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.413897 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" containerName="manager" Jul 09 17:34:40.413910 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.413910 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" containerName="manager" Jul 09 17:34:40.414020 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.413974 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" containerName="manager" Jul 09 17:34:40.416979 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.416958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:40.434183 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.434156 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v"] Jul 09 17:34:40.448030 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.447991 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.607696 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.607665 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q4wm\" (UniqueName: \"kubernetes.io/projected/217b8486-1467-4aa4-8b04-db9a3a7274b3-kube-api-access-6q4wm\") pod \"limitador-operator-controller-manager-859d7b78bd-79h2v\" (UID: \"217b8486-1467-4aa4-8b04-db9a3a7274b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:40.623748 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.623715 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.626037 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.626005 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.708358 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.708257 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff7xw\" (UniqueName: \"kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw\") pod \"72f001a7-94b3-42a5-9175-aa157be455bf\" (UID: \"72f001a7-94b3-42a5-9175-aa157be455bf\") " Jul 09 17:34:40.708563 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.708421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q4wm\" (UniqueName: \"kubernetes.io/projected/217b8486-1467-4aa4-8b04-db9a3a7274b3-kube-api-access-6q4wm\") pod \"limitador-operator-controller-manager-859d7b78bd-79h2v\" (UID: \"217b8486-1467-4aa4-8b04-db9a3a7274b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:40.710634 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.710594 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw" (OuterVolumeSpecName: "kube-api-access-ff7xw") pod "72f001a7-94b3-42a5-9175-aa157be455bf" (UID: "72f001a7-94b3-42a5-9175-aa157be455bf"). InnerVolumeSpecName "kube-api-access-ff7xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:40.716483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.716457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q4wm\" (UniqueName: \"kubernetes.io/projected/217b8486-1467-4aa4-8b04-db9a3a7274b3-kube-api-access-6q4wm\") pod \"limitador-operator-controller-manager-859d7b78bd-79h2v\" (UID: \"217b8486-1467-4aa4-8b04-db9a3a7274b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:40.774545 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.774506 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:40.808906 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.808874 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff7xw\" (UniqueName: \"kubernetes.io/projected/72f001a7-94b3-42a5-9175-aa157be455bf-kube-api-access-ff7xw\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:34:40.908063 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.908028 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v"] Jul 09 17:34:40.911236 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:34:40.911208 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217b8486_1467_4aa4_8b04_db9a3a7274b3.slice/crio-8a2db9d611897cfb54c185e3eaa368c032568c77bf1f4424ebd52c5cecdad8ac WatchSource:0}: Error finding container 8a2db9d611897cfb54c185e3eaa368c032568c77bf1f4424ebd52c5cecdad8ac: Status 404 returned error can't find the container with id 8a2db9d611897cfb54c185e3eaa368c032568c77bf1f4424ebd52c5cecdad8ac Jul 09 17:34:40.947414 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.947384 2577 generic.go:356] "Generic (PLEG): container finished" podID="72f001a7-94b3-42a5-9175-aa157be455bf" containerID="d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660" exitCode=0 Jul 09 17:34:40.947567 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.947442 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.947567 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.947484 2577 scope.go:117] "RemoveContainer" containerID="d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660" Jul 09 17:34:40.948831 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.948805 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" event={"ID":"217b8486-1467-4aa4-8b04-db9a3a7274b3","Type":"ContainerStarted","Data":"8a2db9d611897cfb54c185e3eaa368c032568c77bf1f4424ebd52c5cecdad8ac"} Jul 09 17:34:40.949727 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.949693 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.967607 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.967576 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:40.968213 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.968190 2577 scope.go:117] "RemoveContainer" containerID="d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660" Jul 09 17:34:40.968575 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:40.968553 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660\": container with ID starting with d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660 not found: ID does not exist" containerID="d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660" Jul 09 17:34:40.968663 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:40.968585 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660"} err="failed to get container status \"d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660\": rpc error: code = NotFound desc = could not find container \"d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660\": container with ID starting with d76e8a838837da3e5784d3ebdbea7adade27506dcef29db103709f49f992d660 not found: ID does not exist" Jul 09 17:34:41.955576 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:41.955539 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" event={"ID":"217b8486-1467-4aa4-8b04-db9a3a7274b3","Type":"ContainerStarted","Data":"71d9cf03beac5edc7ad729f322f9e101939a0900783cc2a9821172b2bd2389df"} Jul 09 17:34:41.955959 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:41.955664 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:34:41.976211 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:41.976156 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" podStartSLOduration=1.976139522 podStartE2EDuration="1.976139522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:41.974838088 +0000 UTC m=+424.423890492" watchObservedRunningTime="2026-07-09 17:34:41.976139522 +0000 UTC m=+424.425191924" Jul 09 17:34:41.976866 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:34:41.976836 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-htqln\" is forbidden: User \"system:node:ip-10-0-130-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-12.ec2.internal' and this object" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-htqln" Jul 09 17:34:42.073810 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:42.073779 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f001a7-94b3-42a5-9175-aa157be455bf" path="/var/lib/kubelet/pods/72f001a7-94b3-42a5-9175-aa157be455bf/volumes" Jul 09 17:34:52.962020 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:34:52.961990 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-79h2v" Jul 09 17:35:10.879449 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.879325 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:10.885165 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.885142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:10.887512 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.887490 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 17:35:10.887617 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.887559 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8qdz7\"" Jul 09 17:35:10.890921 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.890897 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:10.942254 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.942214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6fd\" (UniqueName: \"kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:10.942446 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.942268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:10.975214 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:10.975179 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:11.043724 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.043684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:11.043910 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.043775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6fd\" (UniqueName: \"kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:11.044428 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.044408 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:11.051280 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.051256 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6fd\" (UniqueName: \"kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd\") pod \"limitador-limitador-64c94fd64f-x2whl\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:11.196673 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.196591 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:11.329703 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:11.329669 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:11.333063 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:35:11.333035 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697bb48a_9673_4be2_966d_f6777d5af9ff.slice/crio-ef6ba3af2cd4f728f68ba48d89bb430bc21bf5d8f856a5293cd493430e7d66ee WatchSource:0}: Error finding container ef6ba3af2cd4f728f68ba48d89bb430bc21bf5d8f856a5293cd493430e7d66ee: Status 404 returned error can't find the container with id ef6ba3af2cd4f728f68ba48d89bb430bc21bf5d8f856a5293cd493430e7d66ee Jul 09 17:35:12.095469 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:12.095400 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" event={"ID":"697bb48a-9673-4be2-966d-f6777d5af9ff","Type":"ContainerStarted","Data":"ef6ba3af2cd4f728f68ba48d89bb430bc21bf5d8f856a5293cd493430e7d66ee"} Jul 09 17:35:14.106608 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:14.106563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" event={"ID":"697bb48a-9673-4be2-966d-f6777d5af9ff","Type":"ContainerStarted","Data":"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2"} Jul 09 17:35:14.107072 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:14.106703 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:14.121614 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:14.121558 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" podStartSLOduration=1.454644426 podStartE2EDuration="4.12154082s" totalImagesPullingTime="2.666896394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:35:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:35:14.120140339 +0000 UTC m=+456.569192741" watchObservedRunningTime="2026-07-09 17:35:14.12154082 +0000 UTC m=+456.570593221" Jul 09 17:35:25.112802 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:25.112769 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:25.840743 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:25.840712 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:25.840978 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:25.840953 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" podUID="697bb48a-9673-4be2-966d-f6777d5af9ff" containerName="limitador" containerID="cri-o://f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2" gracePeriod=30 Jul 09 17:35:26.385670 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.385645 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:26.475898 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.475818 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp6fd\" (UniqueName: \"kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd\") pod \"697bb48a-9673-4be2-966d-f6777d5af9ff\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " Jul 09 17:35:26.475898 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.475878 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file\") pod \"697bb48a-9673-4be2-966d-f6777d5af9ff\" (UID: \"697bb48a-9673-4be2-966d-f6777d5af9ff\") " Jul 09 17:35:26.476232 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.476198 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file" (OuterVolumeSpecName: "config-file") pod "697bb48a-9673-4be2-966d-f6777d5af9ff" (UID: "697bb48a-9673-4be2-966d-f6777d5af9ff"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:35:26.477874 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.477852 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd" (OuterVolumeSpecName: "kube-api-access-fp6fd") pod "697bb48a-9673-4be2-966d-f6777d5af9ff" (UID: "697bb48a-9673-4be2-966d-f6777d5af9ff"). InnerVolumeSpecName "kube-api-access-fp6fd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:35:26.576914 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.576864 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp6fd\" (UniqueName: \"kubernetes.io/projected/697bb48a-9673-4be2-966d-f6777d5af9ff-kube-api-access-fp6fd\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:35:26.576914 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:26.576907 2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697bb48a-9673-4be2-966d-f6777d5af9ff-config-file\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:35:27.163270 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.163232 2577 generic.go:356] "Generic (PLEG): container finished" podID="697bb48a-9673-4be2-966d-f6777d5af9ff" containerID="f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2" exitCode=0 Jul 09 17:35:27.163270 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.163273 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" Jul 09 17:35:27.163502 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.163323 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" event={"ID":"697bb48a-9673-4be2-966d-f6777d5af9ff","Type":"ContainerDied","Data":"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2"} Jul 09 17:35:27.163502 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.163387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x2whl" event={"ID":"697bb48a-9673-4be2-966d-f6777d5af9ff","Type":"ContainerDied","Data":"ef6ba3af2cd4f728f68ba48d89bb430bc21bf5d8f856a5293cd493430e7d66ee"} Jul 09 17:35:27.163502 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.163412 2577 scope.go:117] "RemoveContainer" containerID="f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2" Jul 09 17:35:27.172541 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.172522 2577 scope.go:117] "RemoveContainer" containerID="f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2" Jul 09 17:35:27.172804 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:35:27.172781 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2\": container with ID starting with f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2 not found: ID does not exist" containerID="f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2" Jul 09 17:35:27.172858 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.172814 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2"} err="failed to get container status \"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2\": rpc error: code = NotFound desc = could not find container \"f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2\": container with ID starting with f29564917b6ff37aaefc741650980cbf59ac94222a7e4f500b694dcec2dbf4a2 not found: ID does not exist" Jul 09 17:35:27.185040 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.185015 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:27.188829 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.188803 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x2whl"] Jul 09 17:35:27.542788 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.542698 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b"] Jul 09 17:35:27.543139 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.543035 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="697bb48a-9673-4be2-966d-f6777d5af9ff" containerName="limitador" Jul 09 17:35:27.543139 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.543046 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="697bb48a-9673-4be2-966d-f6777d5af9ff" containerName="limitador" Jul 09 17:35:27.543139 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.543091 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="697bb48a-9673-4be2-966d-f6777d5af9ff" containerName="limitador" Jul 09 17:35:27.547335 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.547314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.549807 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.549789 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 17:35:27.549943 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.549822 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 17:35:27.549943 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.549863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-94767\"" Jul 09 17:35:27.549943 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.549899 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 17:35:27.554760 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.554737 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b"] Jul 09 17:35:27.689073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.689036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5fa267d3-2459-4dc7-a31d-ded33042ec42-data\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.689073 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.689075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89w46\" (UniqueName: \"kubernetes.io/projected/5fa267d3-2459-4dc7-a31d-ded33042ec42-kube-api-access-89w46\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.790176 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.790142 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5fa267d3-2459-4dc7-a31d-ded33042ec42-data\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.790176 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.790179 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89w46\" (UniqueName: \"kubernetes.io/projected/5fa267d3-2459-4dc7-a31d-ded33042ec42-kube-api-access-89w46\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.790615 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.790595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5fa267d3-2459-4dc7-a31d-ded33042ec42-data\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.799022 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.798961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89w46\" (UniqueName: \"kubernetes.io/projected/5fa267d3-2459-4dc7-a31d-ded33042ec42-kube-api-access-89w46\") pod \"postgres-69fb4594cf-mvb4b\" (UID: \"5fa267d3-2459-4dc7-a31d-ded33042ec42\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.858930 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.858896 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:27.986398 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:27.986350 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b"] Jul 09 17:35:27.989900 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:35:27.989872 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa267d3_2459_4dc7_a31d_ded33042ec42.slice/crio-f9859f23fa6d747a8dfea0d71951f925f451b4db7786e0c3e9ef5b6c91de6aa8 WatchSource:0}: Error finding container f9859f23fa6d747a8dfea0d71951f925f451b4db7786e0c3e9ef5b6c91de6aa8: Status 404 returned error can't find the container with id f9859f23fa6d747a8dfea0d71951f925f451b4db7786e0c3e9ef5b6c91de6aa8 Jul 09 17:35:28.073602 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:28.073573 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="697bb48a-9673-4be2-966d-f6777d5af9ff" path="/var/lib/kubelet/pods/697bb48a-9673-4be2-966d-f6777d5af9ff/volumes" Jul 09 17:35:28.169541 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:28.169503 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" event={"ID":"5fa267d3-2459-4dc7-a31d-ded33042ec42","Type":"ContainerStarted","Data":"f9859f23fa6d747a8dfea0d71951f925f451b4db7786e0c3e9ef5b6c91de6aa8"} Jul 09 17:35:33.196793 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:33.196762 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" event={"ID":"5fa267d3-2459-4dc7-a31d-ded33042ec42","Type":"ContainerStarted","Data":"7af599a4eadaaf8da6aa022a2196b6b0e44937a68ba2d9106c0a3e695860a724"} Jul 09 17:35:33.197180 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:33.196875 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:33.212931 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:33.212856 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" podStartSLOduration=1.473525027 podStartE2EDuration="6.212837266s" totalImagesPullingTime="4.739312239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:35:33.212771692 +0000 UTC m=+475.661824098" watchObservedRunningTime="2026-07-09 17:35:33.212837266 +0000 UTC m=+475.661889688" Jul 09 17:35:39.230762 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:39.230733 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvb4b" Jul 09 17:35:46.263602 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.263562 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:35:46.448855 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.448815 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:35:46.449045 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.448922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.451910 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.451882 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 17:35:46.452142 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.452121 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 17:35:46.452236 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.452215 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qjvll\"" Jul 09 17:35:46.452451 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.452429 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.452582 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.452517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.452627 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.452589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj284\" (UniqueName: \"kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.453233 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.453201 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:35:46.477694 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.477650 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:35:46.477843 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.477754 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.553836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.553836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.553836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553822 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.554142 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj284\" (UniqueName: \"kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.554142 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjvj6\" (UniqueName: \"kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.554142 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.553966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.554566 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.554544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.556169 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.556146 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.564642 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.564620 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj284\" (UniqueName: \"kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284\") pod \"maas-controller-b54df64ff-gk4rb\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.655032 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.654992 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjvj6\" (UniqueName: \"kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.655221 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.655043 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.655221 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.655093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.655887 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.655862 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.657675 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.657655 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.662461 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.662437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjvj6\" (UniqueName: \"kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6\") pod \"maas-controller-57f76b769b-wn859\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.759623 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.759584 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:46.790798 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.790764 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:46.938589 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.938552 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:35:46.941489 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:35:46.941463 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031b7e4d_6c5d_46cb_af94_164e92a1d970.slice/crio-476882c33000908226b55179aa7a6dd1599fc38160e6e738fea2dcba0db01240 WatchSource:0}: Error finding container 476882c33000908226b55179aa7a6dd1599fc38160e6e738fea2dcba0db01240: Status 404 returned error can't find the container with id 476882c33000908226b55179aa7a6dd1599fc38160e6e738fea2dcba0db01240 Jul 09 17:35:46.964098 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:46.964077 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:35:46.965997 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:35:46.965965 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701d9e4f_45ad_462f_8bdb_ac7be26b7e4c.slice/crio-9849a6915456b7920aa30f80d01c46625a71216cc14151f74a1ed4584e1c7251 WatchSource:0}: Error finding container 9849a6915456b7920aa30f80d01c46625a71216cc14151f74a1ed4584e1c7251: Status 404 returned error can't find the container with id 9849a6915456b7920aa30f80d01c46625a71216cc14151f74a1ed4584e1c7251 Jul 09 17:35:47.259718 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:47.259662 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wn859" event={"ID":"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c","Type":"ContainerStarted","Data":"9849a6915456b7920aa30f80d01c46625a71216cc14151f74a1ed4584e1c7251"} Jul 09 17:35:47.260809 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:47.260784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b54df64ff-gk4rb" event={"ID":"031b7e4d-6c5d-46cb-af94-164e92a1d970","Type":"ContainerStarted","Data":"476882c33000908226b55179aa7a6dd1599fc38160e6e738fea2dcba0db01240"} Jul 09 17:35:50.277280 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.277234 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wn859" event={"ID":"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c","Type":"ContainerStarted","Data":"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0"} Jul 09 17:35:50.277762 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.277431 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:35:50.278740 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.278718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b54df64ff-gk4rb" event={"ID":"031b7e4d-6c5d-46cb-af94-164e92a1d970","Type":"ContainerStarted","Data":"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d"} Jul 09 17:35:50.278861 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.278769 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:35:50.296440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.296389 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57f76b769b-wn859" podStartSLOduration=1.440289 podStartE2EDuration="4.296356798s" totalImagesPullingTime="2.856067798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:35:50.294349725 +0000 UTC m=+492.743402127" watchObservedRunningTime="2026-07-09 17:35:50.296356798 +0000 UTC m=+492.745409260" Jul 09 17:35:50.319145 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:35:50.319090 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b54df64ff-gk4rb" podStartSLOduration=1.449933696 podStartE2EDuration="4.319074628s" totalImagesPullingTime="2.869140932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:35:50.317296918 +0000 UTC m=+492.766349317" watchObservedRunningTime="2026-07-09 17:35:50.319074628 +0000 UTC m=+492.768127029" Jul 09 17:36:01.289240 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.289211 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:36:01.289739 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.289287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:36:01.351336 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.351305 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:36:01.351575 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.351520 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b54df64ff-gk4rb" podUID="031b7e4d-6c5d-46cb-af94-164e92a1d970" containerName="manager" containerID="cri-o://9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d" gracePeriod=30 Jul 09 17:36:01.597396 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.597356 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:36:01.671839 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.671804 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj284\" (UniqueName: \"kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284\") pod \"031b7e4d-6c5d-46cb-af94-164e92a1d970\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " Jul 09 17:36:01.672033 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.671854 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca\") pod \"031b7e4d-6c5d-46cb-af94-164e92a1d970\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " Jul 09 17:36:01.672033 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.671923 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert\") pod \"031b7e4d-6c5d-46cb-af94-164e92a1d970\" (UID: \"031b7e4d-6c5d-46cb-af94-164e92a1d970\") " Jul 09 17:36:01.672306 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.672276 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca" (OuterVolumeSpecName: "service-ca") pod "031b7e4d-6c5d-46cb-af94-164e92a1d970" (UID: "031b7e4d-6c5d-46cb-af94-164e92a1d970"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:36:01.674083 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.674051 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284" (OuterVolumeSpecName: "kube-api-access-tj284") pod "031b7e4d-6c5d-46cb-af94-164e92a1d970" (UID: "031b7e4d-6c5d-46cb-af94-164e92a1d970"). InnerVolumeSpecName "kube-api-access-tj284". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:36:01.674183 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.674118 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "031b7e4d-6c5d-46cb-af94-164e92a1d970" (UID: "031b7e4d-6c5d-46cb-af94-164e92a1d970"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:36:01.772865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.772832 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/031b7e4d-6c5d-46cb-af94-164e92a1d970-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:36:01.772865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.772861 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/031b7e4d-6c5d-46cb-af94-164e92a1d970-webhook-cert\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:36:01.772865 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:01.772871 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj284\" (UniqueName: \"kubernetes.io/projected/031b7e4d-6c5d-46cb-af94-164e92a1d970-kube-api-access-tj284\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:36:02.332245 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.332207 2577 generic.go:356] "Generic (PLEG): container finished" podID="031b7e4d-6c5d-46cb-af94-164e92a1d970" containerID="9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d" exitCode=0 Jul 09 17:36:02.332784 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.332280 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b54df64ff-gk4rb" Jul 09 17:36:02.332784 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.332302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b54df64ff-gk4rb" event={"ID":"031b7e4d-6c5d-46cb-af94-164e92a1d970","Type":"ContainerDied","Data":"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d"} Jul 09 17:36:02.332784 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.332399 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b54df64ff-gk4rb" event={"ID":"031b7e4d-6c5d-46cb-af94-164e92a1d970","Type":"ContainerDied","Data":"476882c33000908226b55179aa7a6dd1599fc38160e6e738fea2dcba0db01240"} Jul 09 17:36:02.332784 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.332423 2577 scope.go:117] "RemoveContainer" containerID="9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d" Jul 09 17:36:02.341052 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.341032 2577 scope.go:117] "RemoveContainer" containerID="9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d" Jul 09 17:36:02.341311 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:36:02.341291 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d\": container with ID starting with 9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d not found: ID does not exist" containerID="9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d" Jul 09 17:36:02.341383 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.341317 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d"} err="failed to get container status \"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d\": rpc error: code = NotFound desc = could not find container \"9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d\": container with ID starting with 9e98f642ac9f5706366f5c7285f3ce9d9141ad1620fa064d11c96db9280dd24d not found: ID does not exist" Jul 09 17:36:02.347450 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.347428 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:36:02.350886 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:02.350865 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b54df64ff-gk4rb"] Jul 09 17:36:04.075234 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:04.075201 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="031b7e4d-6c5d-46cb-af94-164e92a1d970" path="/var/lib/kubelet/pods/031b7e4d-6c5d-46cb-af94-164e92a1d970/volumes" Jul 09 17:36:21.876999 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.876960 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b55d88699-mw5hg"] Jul 09 17:36:21.877606 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.877538 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="031b7e4d-6c5d-46cb-af94-164e92a1d970" containerName="manager" Jul 09 17:36:21.877606 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.877559 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="031b7e4d-6c5d-46cb-af94-164e92a1d970" containerName="manager" Jul 09 17:36:21.877722 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.877660 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="031b7e4d-6c5d-46cb-af94-164e92a1d970" containerName="manager" Jul 09 17:36:21.882057 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.882036 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:21.884156 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.884134 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5lf9x\"" Jul 09 17:36:21.884266 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.884157 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 17:36:21.890852 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.890826 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b55d88699-mw5hg"] Jul 09 17:36:21.937832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.937800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8e78416-1b08-466a-8c71-fde54390e0de-maas-api-tls\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:21.937832 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.937836 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4czz\" (UniqueName: \"kubernetes.io/projected/b8e78416-1b08-466a-8c71-fde54390e0de-kube-api-access-m4czz\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:21.938045 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:21.937863 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8e78416-1b08-466a-8c71-fde54390e0de-openshift-service-ca\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.039195 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.039155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8e78416-1b08-466a-8c71-fde54390e0de-openshift-service-ca\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.039386 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.039236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8e78416-1b08-466a-8c71-fde54390e0de-maas-api-tls\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.039386 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.039256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4czz\" (UniqueName: \"kubernetes.io/projected/b8e78416-1b08-466a-8c71-fde54390e0de-kube-api-access-m4czz\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.039852 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.039828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8e78416-1b08-466a-8c71-fde54390e0de-openshift-service-ca\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.041754 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.041727 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8e78416-1b08-466a-8c71-fde54390e0de-maas-api-tls\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.047116 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.047096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4czz\" (UniqueName: \"kubernetes.io/projected/b8e78416-1b08-466a-8c71-fde54390e0de-kube-api-access-m4czz\") pod \"maas-api-b55d88699-mw5hg\" (UID: \"b8e78416-1b08-466a-8c71-fde54390e0de\") " pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.194492 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.194382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:22.366629 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.366521 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b55d88699-mw5hg"] Jul 09 17:36:22.368903 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:36:22.368877 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e78416_1b08_466a_8c71_fde54390e0de.slice/crio-e7f0814aac5f367f52c994e667bb77e8952735f76a18881a1a4fabf057ce28b1 WatchSource:0}: Error finding container e7f0814aac5f367f52c994e667bb77e8952735f76a18881a1a4fabf057ce28b1: Status 404 returned error can't find the container with id e7f0814aac5f367f52c994e667bb77e8952735f76a18881a1a4fabf057ce28b1 Jul 09 17:36:22.422184 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:22.422147 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" event={"ID":"b8e78416-1b08-466a-8c71-fde54390e0de","Type":"ContainerStarted","Data":"e7f0814aac5f367f52c994e667bb77e8952735f76a18881a1a4fabf057ce28b1"} Jul 09 17:36:24.434120 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:24.434085 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" event={"ID":"b8e78416-1b08-466a-8c71-fde54390e0de","Type":"ContainerStarted","Data":"14ff7335085a9a0f71996742aa10b5aedf6648c413beec2db9b1970f454b3970"} Jul 09 17:36:24.434594 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:24.434148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:24.448955 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:24.448901 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" podStartSLOduration=1.492155688 podStartE2EDuration="3.448882551s" totalImagesPullingTime="1.956726863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:24.448669516 +0000 UTC m=+526.897721927" watchObservedRunningTime="2026-07-09 17:36:24.448882551 +0000 UTC m=+526.897934953" Jul 09 17:36:30.444383 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:30.444296 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b55d88699-mw5hg" Jul 09 17:36:38.587423 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.587387 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh"] Jul 09 17:36:38.591902 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.591886 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.596058 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.596024 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 17:36:38.596228 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.596024 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 17:36:38.596228 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.596095 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c6gkt\"" Jul 09 17:36:38.596361 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.596094 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 17:36:38.598415 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.598392 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh"] Jul 09 17:36:38.688999 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.688961 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.689208 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.689016 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxgsj\" (UniqueName: \"kubernetes.io/projected/37f16509-ee31-495a-a334-434be73a1b9e-kube-api-access-nxgsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.689208 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.689101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.689208 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.689156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.689208 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.689195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f16509-ee31-495a-a334-434be73a1b9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.689447 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.689220 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790464 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790683 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790503 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxgsj\" (UniqueName: \"kubernetes.io/projected/37f16509-ee31-495a-a334-434be73a1b9e-kube-api-access-nxgsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790683 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790556 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790683 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790683 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f16509-ee31-495a-a334-434be73a1b9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790683 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790927 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790889 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.790927 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790914 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.791032 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.790960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.792855 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.792834 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f16509-ee31-495a-a334-434be73a1b9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.793242 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.793225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f16509-ee31-495a-a334-434be73a1b9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.798320 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.798297 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxgsj\" (UniqueName: \"kubernetes.io/projected/37f16509-ee31-495a-a334-434be73a1b9e-kube-api-access-nxgsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh\" (UID: \"37f16509-ee31-495a-a334-434be73a1b9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:38.903537 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:38.903443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:39.032328 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:39.032295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh"] Jul 09 17:36:39.035431 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:36:39.035395 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f16509_ee31_495a_a334_434be73a1b9e.slice/crio-8da8196431bfe8cf7ce333c1f2d445d9c85e6f5a5e1ad1deab4270bd7cb24cf2 WatchSource:0}: Error finding container 8da8196431bfe8cf7ce333c1f2d445d9c85e6f5a5e1ad1deab4270bd7cb24cf2: Status 404 returned error can't find the container with id 8da8196431bfe8cf7ce333c1f2d445d9c85e6f5a5e1ad1deab4270bd7cb24cf2 Jul 09 17:36:39.506273 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:39.506239 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" event={"ID":"37f16509-ee31-495a-a334-434be73a1b9e","Type":"ContainerStarted","Data":"8da8196431bfe8cf7ce333c1f2d445d9c85e6f5a5e1ad1deab4270bd7cb24cf2"} Jul 09 17:36:40.512562 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:40.512473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" event={"ID":"37f16509-ee31-495a-a334-434be73a1b9e","Type":"ContainerStarted","Data":"57d4d6ff3676db3be32ca4cba2d384179dd5151543da980af11d5ea39e8fc8c5"} Jul 09 17:36:40.528565 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:40.528508 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" podStartSLOduration=1.31822857 podStartE2EDuration="2.528489717s" totalImagesPullingTime="1.210261147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:40.52814151 +0000 UTC m=+542.977193931" watchObservedRunningTime="2026-07-09 17:36:40.528489717 +0000 UTC m=+542.977542118" Jul 09 17:36:48.904415 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:48.904381 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:48.904891 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:48.904478 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:48.916609 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:48.916579 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:36:49.565488 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:36:49.565456 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh" Jul 09 17:37:13.871180 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.871146 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl"] Jul 09 17:37:13.877603 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.877576 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.877931 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.877905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.878013 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.877951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.878074 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.878016 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwhkd\" (UniqueName: \"kubernetes.io/projected/a46f8d48-a311-495c-a907-e384258ecfa9-kube-api-access-vwhkd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.878132 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.878108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.878190 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.878139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46f8d48-a311-495c-a907-e384258ecfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.878262 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.878248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.880051 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.880029 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 17:37:13.882950 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.882929 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl"] Jul 09 17:37:13.978585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.978585 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.978803 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.978803 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwhkd\" (UniqueName: \"kubernetes.io/projected/a46f8d48-a311-495c-a907-e384258ecfa9-kube-api-access-vwhkd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.978803 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978666 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.978803 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46f8d48-a311-495c-a907-e384258ecfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.979036 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.978991 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.979097 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.979051 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.979171 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.979110 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.980856 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.980837 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a46f8d48-a311-495c-a907-e384258ecfa9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.981194 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.981179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a46f8d48-a311-495c-a907-e384258ecfa9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:13.986905 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:13.986884 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwhkd\" (UniqueName: \"kubernetes.io/projected/a46f8d48-a311-495c-a907-e384258ecfa9-kube-api-access-vwhkd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nzckl\" (UID: \"a46f8d48-a311-495c-a907-e384258ecfa9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:14.189109 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:14.189023 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:14.327040 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:14.327008 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl"] Jul 09 17:37:14.330418 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:37:14.330350 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46f8d48_a311_495c_a907_e384258ecfa9.slice/crio-a92191402347408dddec599fcf5636ba782f63fd066b62558b44d076f151026a WatchSource:0}: Error finding container a92191402347408dddec599fcf5636ba782f63fd066b62558b44d076f151026a: Status 404 returned error can't find the container with id a92191402347408dddec599fcf5636ba782f63fd066b62558b44d076f151026a Jul 09 17:37:14.676702 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:14.676668 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" event={"ID":"a46f8d48-a311-495c-a907-e384258ecfa9","Type":"ContainerStarted","Data":"21eda4c50884064356c2811436d9ca09ef763ce198200b80ba6891c82e466152"} Jul 09 17:37:14.676877 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:14.676751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" event={"ID":"a46f8d48-a311-495c-a907-e384258ecfa9","Type":"ContainerStarted","Data":"a92191402347408dddec599fcf5636ba782f63fd066b62558b44d076f151026a"} Jul 09 17:37:14.693206 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:14.693153 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" podStartSLOduration=1.514392134 podStartE2EDuration="1.693136312s" totalImagesPullingTime="178.744178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:37:14.692296032 +0000 UTC m=+577.141348434" watchObservedRunningTime="2026-07-09 17:37:14.693136312 +0000 UTC m=+577.142188713" Jul 09 17:37:24.189583 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:24.189529 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:24.190067 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:24.189622 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:24.202084 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:24.202054 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:24.734808 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:24.734779 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nzckl" Jul 09 17:37:29.870582 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.870548 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh"] Jul 09 17:37:29.874235 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.874214 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.876419 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.876392 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 17:37:29.883592 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.883565 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh"] Jul 09 17:37:29.896264 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx25x\" (UniqueName: \"kubernetes.io/projected/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-kube-api-access-tx25x\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.896426 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.896426 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.896501 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.896501 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.896567 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.896510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997126 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997312 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx25x\" (UniqueName: \"kubernetes.io/projected/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-kube-api-access-tx25x\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997312 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997269 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997483 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997642 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997699 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997657 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.997765 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.997746 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.999735 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.999713 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:29.999886 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:29.999869 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:30.003894 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:30.003870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx25x\" (UniqueName: \"kubernetes.io/projected/2fa76ab5-ebbe-480e-923e-eacc1badd7f6-kube-api-access-tx25x\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh\" (UID: \"2fa76ab5-ebbe-480e-923e-eacc1badd7f6\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:30.185406 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:30.185292 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:30.320607 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:30.320534 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh"] Jul 09 17:37:30.323377 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:37:30.323337 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa76ab5_ebbe_480e_923e_eacc1badd7f6.slice/crio-6b60501e7acc438decc6553a457c843f4b5993515dbc8f569ecc6f728b71f92e WatchSource:0}: Error finding container 6b60501e7acc438decc6553a457c843f4b5993515dbc8f569ecc6f728b71f92e: Status 404 returned error can't find the container with id 6b60501e7acc438decc6553a457c843f4b5993515dbc8f569ecc6f728b71f92e Jul 09 17:37:30.752024 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:30.751986 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" event={"ID":"2fa76ab5-ebbe-480e-923e-eacc1badd7f6","Type":"ContainerStarted","Data":"cfbad9cc26081c31c33aa49ab9a36109590b8a17acedb066b5dd035de8321f27"} Jul 09 17:37:30.752024 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:30.752037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" event={"ID":"2fa76ab5-ebbe-480e-923e-eacc1badd7f6","Type":"ContainerStarted","Data":"6b60501e7acc438decc6553a457c843f4b5993515dbc8f569ecc6f728b71f92e"} Jul 09 17:37:38.008897 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:38.008866 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:37:38.009399 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:38.009361 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:37:40.186380 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:40.186329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:40.186380 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:40.186390 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:40.198783 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:40.198754 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:37:40.214514 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:40.214464 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" podStartSLOduration=11.027638864 podStartE2EDuration="11.214449518s" totalImagesPullingTime="186.810654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:37:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:37:30.772781906 +0000 UTC m=+593.221834309" watchObservedRunningTime="2026-07-09 17:37:40.214449518 +0000 UTC m=+602.663501920" Jul 09 17:37:40.808935 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:37:40.808899 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh" Jul 09 17:40:11.438985 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.438950 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:40:11.439538 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.439284 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57f76b769b-wn859" podUID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" containerName="manager" containerID="cri-o://60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0" gracePeriod=30 Jul 09 17:40:11.681308 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.681281 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:40:11.719952 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.719873 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca\") pod \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " Jul 09 17:40:11.720082 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.719978 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert\") pod \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " Jul 09 17:40:11.720082 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.720004 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjvj6\" (UniqueName: \"kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6\") pod \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\" (UID: \"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c\") " Jul 09 17:40:11.720319 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.720294 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca" (OuterVolumeSpecName: "service-ca") pod "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" (UID: "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:40:11.722028 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.721997 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" (UID: "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:40:11.722140 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.722117 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6" (OuterVolumeSpecName: "kube-api-access-qjvj6") pod "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" (UID: "701d9e4f-45ad-462f-8bdb-ac7be26b7e4c"). InnerVolumeSpecName "kube-api-access-qjvj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:40:11.821420 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.821361 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-webhook-cert\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:40:11.821420 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.821413 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjvj6\" (UniqueName: \"kubernetes.io/projected/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-kube-api-access-qjvj6\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:40:11.821420 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:11.821422 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:40:12.486850 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.486768 2577 generic.go:356] "Generic (PLEG): container finished" podID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" containerID="60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0" exitCode=0 Jul 09 17:40:12.486850 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.486808 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wn859" Jul 09 17:40:12.486850 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.486813 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wn859" event={"ID":"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c","Type":"ContainerDied","Data":"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0"} Jul 09 17:40:12.487355 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.486854 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wn859" event={"ID":"701d9e4f-45ad-462f-8bdb-ac7be26b7e4c","Type":"ContainerDied","Data":"9849a6915456b7920aa30f80d01c46625a71216cc14151f74a1ed4584e1c7251"} Jul 09 17:40:12.487355 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.486876 2577 scope.go:117] "RemoveContainer" containerID="60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0" Jul 09 17:40:12.495514 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.495492 2577 scope.go:117] "RemoveContainer" containerID="60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0" Jul 09 17:40:12.495760 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:40:12.495740 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0\": container with ID starting with 60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0 not found: ID does not exist" containerID="60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0" Jul 09 17:40:12.495802 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.495770 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0"} err="failed to get container status \"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0\": rpc error: code = NotFound desc = could not find container \"60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0\": container with ID starting with 60f2d1f16e83314f8d911f85fd02807a9ab5db4f0c7afcced22ca23c41fb03e0 not found: ID does not exist" Jul 09 17:40:12.503546 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.503524 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:40:12.509065 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:12.509043 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wn859"] Jul 09 17:40:13.292489 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.292452 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57f76b769b-wj56k"] Jul 09 17:40:13.292809 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.292797 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" containerName="manager" Jul 09 17:40:13.292851 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.292811 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" containerName="manager" Jul 09 17:40:13.292891 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.292882 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" containerName="manager" Jul 09 17:40:13.297080 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.297063 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.299682 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.299662 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 17:40:13.299878 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.299863 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 17:40:13.300466 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.300450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qjvll\"" Jul 09 17:40:13.306337 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.306315 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wj56k"] Jul 09 17:40:13.335444 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.335417 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-webhook-cert\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.335596 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.335457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-service-ca\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.335596 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.335487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdxj5\" (UniqueName: \"kubernetes.io/projected/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-kube-api-access-wdxj5\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.436564 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.436533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-webhook-cert\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.436730 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.436574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-service-ca\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.436730 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.436601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdxj5\" (UniqueName: \"kubernetes.io/projected/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-kube-api-access-wdxj5\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.437177 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.437151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-service-ca\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.438895 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.438872 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-webhook-cert\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.444307 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.444290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdxj5\" (UniqueName: \"kubernetes.io/projected/6015bcce-4bf6-4c3c-82b4-e81c7cec89b3-kube-api-access-wdxj5\") pod \"maas-controller-57f76b769b-wj56k\" (UID: \"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3\") " pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.607434 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.607401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:13.738665 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.738636 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57f76b769b-wj56k"] Jul 09 17:40:13.742146 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:40:13.742121 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6015bcce_4bf6_4c3c_82b4_e81c7cec89b3.slice/crio-6731fa69cf3f5573512bdd9467e0dd17398847280eea0f095037d742d666615d WatchSource:0}: Error finding container 6731fa69cf3f5573512bdd9467e0dd17398847280eea0f095037d742d666615d: Status 404 returned error can't find the container with id 6731fa69cf3f5573512bdd9467e0dd17398847280eea0f095037d742d666615d Jul 09 17:40:13.743536 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:13.743520 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:40:14.074506 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:14.074483 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="701d9e4f-45ad-462f-8bdb-ac7be26b7e4c" path="/var/lib/kubelet/pods/701d9e4f-45ad-462f-8bdb-ac7be26b7e4c/volumes" Jul 09 17:40:14.499822 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:14.499734 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wj56k" event={"ID":"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3","Type":"ContainerStarted","Data":"3c357346c30b8ddaf008af3d5b3d73b73bf33981b9bd5715ca1d16aae47057d4"} Jul 09 17:40:14.499822 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:14.499786 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57f76b769b-wj56k" event={"ID":"6015bcce-4bf6-4c3c-82b4-e81c7cec89b3","Type":"ContainerStarted","Data":"6731fa69cf3f5573512bdd9467e0dd17398847280eea0f095037d742d666615d"} Jul 09 17:40:14.500006 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:14.499830 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:40:14.516272 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:14.516222 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57f76b769b-wj56k" podStartSLOduration=1.03595446 podStartE2EDuration="1.516206713s" totalImagesPullingTime="480.252253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:14.515085611 +0000 UTC m=+756.964138032" watchObservedRunningTime="2026-07-09 17:40:14.516206713 +0000 UTC m=+756.965259114" Jul 09 17:40:25.509553 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:40:25.509524 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57f76b769b-wj56k" Jul 09 17:42:38.040078 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:42:38.040001 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:42:38.041719 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:42:38.041697 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:47:38.070026 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:47:38.069995 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:47:38.072736 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:47:38.072712 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:52:38.104008 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:38.103974 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:52:38.108253 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:38.108227 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:52:42.776019 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.775987 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:42.779437 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.779415 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:42.781806 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.781786 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-49f4e1e1-serving-cert\"" Jul 09 17:52:42.788201 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.788176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:42.903442 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.903408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:42.903617 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.903521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:42.903617 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:42.903562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qqdf\" (UniqueName: \"kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.004527 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.004496 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.004708 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.004538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qqdf\" (UniqueName: \"kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.004708 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.004584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.005122 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.005096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.006816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.006796 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.012744 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.012712 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qqdf\" (UniqueName: \"kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf\") pod \"maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.090356 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.090320 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.235891 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.235866 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:43.238438 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:52:43.238408 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552ac32d_582d_4d09_b67d_23c5fec33365.slice/crio-c7ee8688a801366a4588a2edc1ee535b78759f71b551ccc7b9b7c8ce117bb3c3 WatchSource:0}: Error finding container c7ee8688a801366a4588a2edc1ee535b78759f71b551ccc7b9b7c8ce117bb3c3: Status 404 returned error can't find the container with id c7ee8688a801366a4588a2edc1ee535b78759f71b551ccc7b9b7c8ce117bb3c3 Jul 09 17:52:43.239833 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.239811 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:52:43.955770 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.955737 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" event={"ID":"552ac32d-582d-4d09-b67d-23c5fec33365","Type":"ContainerStarted","Data":"f3120463c912402e49e6a0d7d1cad25013a22e328e60f23f7eac630e9623978a"} Jul 09 17:52:43.956131 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.955787 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" event={"ID":"552ac32d-582d-4d09-b67d-23c5fec33365","Type":"ContainerStarted","Data":"c7ee8688a801366a4588a2edc1ee535b78759f71b551ccc7b9b7c8ce117bb3c3"} Jul 09 17:52:43.956131 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.955819 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:43.972600 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:43.972508 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" podStartSLOduration=1.499821191 podStartE2EDuration="1.972488909s" totalImagesPullingTime="472.667718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:52:43.97196576 +0000 UTC m=+1506.421018187" watchObservedRunningTime="2026-07-09 17:52:43.972488909 +0000 UTC m=+1506.421541310" Jul 09 17:52:49.966048 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:49.966014 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:50.854183 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:50.854147 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:50.854521 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:50.854335 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" podUID="552ac32d-582d-4d09-b67d-23c5fec33365" containerName="maas-api" containerID="cri-o://f3120463c912402e49e6a0d7d1cad25013a22e328e60f23f7eac630e9623978a" gracePeriod=30 Jul 09 17:52:50.989397 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:50.989343 2577 generic.go:356] "Generic (PLEG): container finished" podID="552ac32d-582d-4d09-b67d-23c5fec33365" containerID="f3120463c912402e49e6a0d7d1cad25013a22e328e60f23f7eac630e9623978a" exitCode=0 Jul 09 17:52:50.989813 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:50.989400 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" event={"ID":"552ac32d-582d-4d09-b67d-23c5fec33365","Type":"ContainerDied","Data":"f3120463c912402e49e6a0d7d1cad25013a22e328e60f23f7eac630e9623978a"} Jul 09 17:52:51.103074 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.103049 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:51.175204 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.175125 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca\") pod \"552ac32d-582d-4d09-b67d-23c5fec33365\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " Jul 09 17:52:51.175204 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.175163 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls\") pod \"552ac32d-582d-4d09-b67d-23c5fec33365\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " Jul 09 17:52:51.175441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.175205 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qqdf\" (UniqueName: \"kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf\") pod \"552ac32d-582d-4d09-b67d-23c5fec33365\" (UID: \"552ac32d-582d-4d09-b67d-23c5fec33365\") " Jul 09 17:52:51.175575 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.175553 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "552ac32d-582d-4d09-b67d-23c5fec33365" (UID: "552ac32d-582d-4d09-b67d-23c5fec33365"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:52:51.177404 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.177356 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "552ac32d-582d-4d09-b67d-23c5fec33365" (UID: "552ac32d-582d-4d09-b67d-23c5fec33365"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:52:51.177516 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.177406 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf" (OuterVolumeSpecName: "kube-api-access-7qqdf") pod "552ac32d-582d-4d09-b67d-23c5fec33365" (UID: "552ac32d-582d-4d09-b67d-23c5fec33365"). InnerVolumeSpecName "kube-api-access-7qqdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:52:51.276525 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.276486 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552ac32d-582d-4d09-b67d-23c5fec33365-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:52:51.276525 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.276516 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552ac32d-582d-4d09-b67d-23c5fec33365-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:52:51.276525 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.276530 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qqdf\" (UniqueName: \"kubernetes.io/projected/552ac32d-582d-4d09-b67d-23c5fec33365-kube-api-access-7qqdf\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:52:51.994941 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.994914 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" Jul 09 17:52:51.995395 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.994913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6" event={"ID":"552ac32d-582d-4d09-b67d-23c5fec33365","Type":"ContainerDied","Data":"c7ee8688a801366a4588a2edc1ee535b78759f71b551ccc7b9b7c8ce117bb3c3"} Jul 09 17:52:51.995395 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:51.995045 2577 scope.go:117] "RemoveContainer" containerID="f3120463c912402e49e6a0d7d1cad25013a22e328e60f23f7eac630e9623978a" Jul 09 17:52:52.014316 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:52.014292 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:52.016466 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:52.016448 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49f4e1e1-68cc46c649-ld6q6"] Jul 09 17:52:52.074732 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:52.074702 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="552ac32d-582d-4d09-b67d-23c5fec33365" path="/var/lib/kubelet/pods/552ac32d-582d-4d09-b67d-23c5fec33365/volumes" Jul 09 17:52:58.378798 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.378765 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:52:58.379289 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.379275 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="552ac32d-582d-4d09-b67d-23c5fec33365" containerName="maas-api" Jul 09 17:52:58.379336 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.379292 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="552ac32d-582d-4d09-b67d-23c5fec33365" containerName="maas-api" Jul 09 17:52:58.379407 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.379396 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="552ac32d-582d-4d09-b67d-23c5fec33365" containerName="maas-api" Jul 09 17:52:58.383857 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.383835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.386045 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.386025 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7e0dcb49-serving-cert\"" Jul 09 17:52:58.391767 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.391744 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:52:58.548728 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.548695 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6n8r\" (UniqueName: \"kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.548728 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.548731 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.548926 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.548825 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.650272 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.650195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6n8r\" (UniqueName: \"kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.650272 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.650237 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.650482 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.650284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.650889 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.650863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.652665 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.652643 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.658839 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.658821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6n8r\" (UniqueName: \"kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r\") pod \"maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.697851 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.697830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:52:58.828915 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:58.828891 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:52:58.831400 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:52:58.831349 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf13f41_d5b3_4baa_8896_c861390f5404.slice/crio-3157b3737dc4d10ae1a8704be2a40105c5310e40bbe9bedcecfbf9ef35e6b947 WatchSource:0}: Error finding container 3157b3737dc4d10ae1a8704be2a40105c5310e40bbe9bedcecfbf9ef35e6b947: Status 404 returned error can't find the container with id 3157b3737dc4d10ae1a8704be2a40105c5310e40bbe9bedcecfbf9ef35e6b947 Jul 09 17:52:59.031247 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:52:59.031163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" event={"ID":"6cf13f41-d5b3-4baa-8896-c861390f5404","Type":"ContainerStarted","Data":"3157b3737dc4d10ae1a8704be2a40105c5310e40bbe9bedcecfbf9ef35e6b947"} Jul 09 17:53:00.037562 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:00.037531 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" event={"ID":"6cf13f41-d5b3-4baa-8896-c861390f5404","Type":"ContainerStarted","Data":"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e"} Jul 09 17:53:00.037960 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:00.037660 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:53:00.055168 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:00.055123 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" podStartSLOduration=1.702092352 podStartE2EDuration="2.055109677s" totalImagesPullingTime="353.017325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:53:00.054210574 +0000 UTC m=+1522.503263019" watchObservedRunningTime="2026-07-09 17:53:00.055109677 +0000 UTC m=+1522.504162076" Jul 09 17:53:06.054876 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.054847 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:53:06.308092 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.307991 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:53:06.308291 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.308262 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" podUID="6cf13f41-d5b3-4baa-8896-c861390f5404" containerName="maas-api" containerID="cri-o://e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e" gracePeriod=30 Jul 09 17:53:06.552160 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.552137 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:53:06.617210 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.617180 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls\") pod \"6cf13f41-d5b3-4baa-8896-c861390f5404\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " Jul 09 17:53:06.617349 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.617255 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6n8r\" (UniqueName: \"kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r\") pod \"6cf13f41-d5b3-4baa-8896-c861390f5404\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " Jul 09 17:53:06.617349 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.617301 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca\") pod \"6cf13f41-d5b3-4baa-8896-c861390f5404\" (UID: \"6cf13f41-d5b3-4baa-8896-c861390f5404\") " Jul 09 17:53:06.617685 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.617661 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6cf13f41-d5b3-4baa-8896-c861390f5404" (UID: "6cf13f41-d5b3-4baa-8896-c861390f5404"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:53:06.619203 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.619181 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r" (OuterVolumeSpecName: "kube-api-access-v6n8r") pod "6cf13f41-d5b3-4baa-8896-c861390f5404" (UID: "6cf13f41-d5b3-4baa-8896-c861390f5404"). InnerVolumeSpecName "kube-api-access-v6n8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:53:06.619267 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.619248 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6cf13f41-d5b3-4baa-8896-c861390f5404" (UID: "6cf13f41-d5b3-4baa-8896-c861390f5404"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:53:06.718866 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.718830 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6n8r\" (UniqueName: \"kubernetes.io/projected/6cf13f41-d5b3-4baa-8896-c861390f5404-kube-api-access-v6n8r\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:06.718866 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.718868 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cf13f41-d5b3-4baa-8896-c861390f5404-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:06.719050 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:06.718881 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cf13f41-d5b3-4baa-8896-c861390f5404-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:07.079265 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.079234 2577 generic.go:356] "Generic (PLEG): container finished" podID="6cf13f41-d5b3-4baa-8896-c861390f5404" containerID="e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e" exitCode=0 Jul 09 17:53:07.079648 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.079277 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" Jul 09 17:53:07.079648 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.079320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" event={"ID":"6cf13f41-d5b3-4baa-8896-c861390f5404","Type":"ContainerDied","Data":"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e"} Jul 09 17:53:07.079648 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.079384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h" event={"ID":"6cf13f41-d5b3-4baa-8896-c861390f5404","Type":"ContainerDied","Data":"3157b3737dc4d10ae1a8704be2a40105c5310e40bbe9bedcecfbf9ef35e6b947"} Jul 09 17:53:07.079648 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.079413 2577 scope.go:117] "RemoveContainer" containerID="e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e" Jul 09 17:53:07.087945 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.087930 2577 scope.go:117] "RemoveContainer" containerID="e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e" Jul 09 17:53:07.088173 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:53:07.088157 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e\": container with ID starting with e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e not found: ID does not exist" containerID="e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e" Jul 09 17:53:07.088212 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.088180 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e"} err="failed to get container status \"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e\": rpc error: code = NotFound desc = could not find container \"e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e\": container with ID starting with e6db20c40a19e4a16a0d2d457b2f15742dcf41325156759051569acc6322760e not found: ID does not exist" Jul 09 17:53:07.100718 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.100693 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:53:07.104298 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:07.104279 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7e0dcb49-6f6cbcd4d4-x8z2h"] Jul 09 17:53:08.074511 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:08.074472 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cf13f41-d5b3-4baa-8896-c861390f5404" path="/var/lib/kubelet/pods/6cf13f41-d5b3-4baa-8896-c861390f5404/volumes" Jul 09 17:53:14.434924 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.434891 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:14.435323 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.435247 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cf13f41-d5b3-4baa-8896-c861390f5404" containerName="maas-api" Jul 09 17:53:14.435323 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.435258 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cf13f41-d5b3-4baa-8896-c861390f5404" containerName="maas-api" Jul 09 17:53:14.435323 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.435322 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cf13f41-d5b3-4baa-8896-c861390f5404" containerName="maas-api" Jul 09 17:53:14.438452 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.438429 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.440746 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.440725 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1608c3e9-serving-cert\"" Jul 09 17:53:14.451327 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.451287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:14.490175 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.490137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.490359 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.490199 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.490359 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.490231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gfp5\" (UniqueName: \"kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.590986 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.590951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.591186 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.591006 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.591186 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.591030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gfp5\" (UniqueName: \"kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.591726 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.591706 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.593427 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.593403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.599526 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.599502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gfp5\" (UniqueName: \"kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5\") pod \"maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.752836 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.752738 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:14.888441 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:14.888324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:14.891188 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:53:14.891157 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696a50e5_27f5_4cea_b74a_55e50bd473a4.slice/crio-9a8746ff1057427ce44a859591db2092647082270ded0d9a6eb949d2f5471d30 WatchSource:0}: Error finding container 9a8746ff1057427ce44a859591db2092647082270ded0d9a6eb949d2f5471d30: Status 404 returned error can't find the container with id 9a8746ff1057427ce44a859591db2092647082270ded0d9a6eb949d2f5471d30 Jul 09 17:53:15.123275 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:15.123224 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" event={"ID":"696a50e5-27f5-4cea-b74a-55e50bd473a4","Type":"ContainerStarted","Data":"9a8746ff1057427ce44a859591db2092647082270ded0d9a6eb949d2f5471d30"} Jul 09 17:53:16.129296 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:16.129256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" event={"ID":"696a50e5-27f5-4cea-b74a-55e50bd473a4","Type":"ContainerStarted","Data":"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150"} Jul 09 17:53:16.129795 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:16.129403 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:16.146739 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:16.146677 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" podStartSLOduration=1.7884448590000002 podStartE2EDuration="2.146656468s" totalImagesPullingTime="358.211609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:53:16.144745798 +0000 UTC m=+1538.593798200" watchObservedRunningTime="2026-07-09 17:53:16.146656468 +0000 UTC m=+1538.595708871" Jul 09 17:53:21.822404 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:21.822346 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:21.822884 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:21.822706 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" podUID="696a50e5-27f5-4cea-b74a-55e50bd473a4" containerName="maas-api" containerID="cri-o://5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150" gracePeriod=30 Jul 09 17:53:21.829582 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:21.829556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:22.078095 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.078070 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:22.155892 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.155864 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gfp5\" (UniqueName: \"kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5\") pod \"696a50e5-27f5-4cea-b74a-55e50bd473a4\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " Jul 09 17:53:22.156090 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.155907 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls\") pod \"696a50e5-27f5-4cea-b74a-55e50bd473a4\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " Jul 09 17:53:22.156090 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.155936 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca\") pod \"696a50e5-27f5-4cea-b74a-55e50bd473a4\" (UID: \"696a50e5-27f5-4cea-b74a-55e50bd473a4\") " Jul 09 17:53:22.156459 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.156432 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "696a50e5-27f5-4cea-b74a-55e50bd473a4" (UID: "696a50e5-27f5-4cea-b74a-55e50bd473a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:53:22.157297 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.157270 2577 generic.go:356] "Generic (PLEG): container finished" podID="696a50e5-27f5-4cea-b74a-55e50bd473a4" containerID="5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150" exitCode=0 Jul 09 17:53:22.157440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.157341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" event={"ID":"696a50e5-27f5-4cea-b74a-55e50bd473a4","Type":"ContainerDied","Data":"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150"} Jul 09 17:53:22.157440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.157348 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" Jul 09 17:53:22.157440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.157414 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9" event={"ID":"696a50e5-27f5-4cea-b74a-55e50bd473a4","Type":"ContainerDied","Data":"9a8746ff1057427ce44a859591db2092647082270ded0d9a6eb949d2f5471d30"} Jul 09 17:53:22.157440 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.157440 2577 scope.go:117] "RemoveContainer" containerID="5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150" Jul 09 17:53:22.158262 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.158236 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "696a50e5-27f5-4cea-b74a-55e50bd473a4" (UID: "696a50e5-27f5-4cea-b74a-55e50bd473a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:53:22.158344 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.158279 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5" (OuterVolumeSpecName: "kube-api-access-9gfp5") pod "696a50e5-27f5-4cea-b74a-55e50bd473a4" (UID: "696a50e5-27f5-4cea-b74a-55e50bd473a4"). InnerVolumeSpecName "kube-api-access-9gfp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:53:22.178612 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.178591 2577 scope.go:117] "RemoveContainer" containerID="5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150" Jul 09 17:53:22.178932 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:53:22.178899 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150\": container with ID starting with 5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150 not found: ID does not exist" containerID="5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150" Jul 09 17:53:22.178981 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.178936 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150"} err="failed to get container status \"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150\": rpc error: code = NotFound desc = could not find container \"5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150\": container with ID starting with 5afc001897685a18c52f811dd9e6ce24c9842e0be137f761ced878caea12e150 not found: ID does not exist" Jul 09 17:53:22.256763 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.256727 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gfp5\" (UniqueName: \"kubernetes.io/projected/696a50e5-27f5-4cea-b74a-55e50bd473a4-kube-api-access-9gfp5\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:22.256763 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.256754 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/696a50e5-27f5-4cea-b74a-55e50bd473a4-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:22.256763 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.256764 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/696a50e5-27f5-4cea-b74a-55e50bd473a4-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:53:22.478825 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.478796 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:22.484970 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:22.484941 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1608c3e9-d7477f4d-8ghd9"] Jul 09 17:53:24.073932 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:53:24.073896 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="696a50e5-27f5-4cea-b74a-55e50bd473a4" path="/var/lib/kubelet/pods/696a50e5-27f5-4cea-b74a-55e50bd473a4/volumes" Jul 09 17:56:53.613591 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.613559 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:56:53.614064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.613917 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="696a50e5-27f5-4cea-b74a-55e50bd473a4" containerName="maas-api" Jul 09 17:56:53.614064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.613929 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="696a50e5-27f5-4cea-b74a-55e50bd473a4" containerName="maas-api" Jul 09 17:56:53.614064 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.613985 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="696a50e5-27f5-4cea-b74a-55e50bd473a4" containerName="maas-api" Jul 09 17:56:53.617211 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.617192 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.620317 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.620297 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9bc868f0-serving-cert\"" Jul 09 17:56:53.638605 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.638574 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:56:53.724919 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.724876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf8zh\" (UniqueName: \"kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.724919 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.724926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.725140 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.725037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.825946 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.825901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf8zh\" (UniqueName: \"kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.825946 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.825950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.826209 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.825991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.826687 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.826666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.828499 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.828483 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.841250 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.841218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf8zh\" (UniqueName: \"kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh\") pod \"maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:53.927722 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:53.927629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:54.078816 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:54.078786 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:56:54.082377 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:56:54.082333 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49a149f_b69c_4af0_8ecc_0589eaf6a343.slice/crio-6a3f73f69cdeeb72610f85ee3dd31570878455d0dcbaa6aaef3926487412784a WatchSource:0}: Error finding container 6a3f73f69cdeeb72610f85ee3dd31570878455d0dcbaa6aaef3926487412784a: Status 404 returned error can't find the container with id 6a3f73f69cdeeb72610f85ee3dd31570878455d0dcbaa6aaef3926487412784a Jul 09 17:56:54.137830 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:54.137792 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" event={"ID":"d49a149f-b69c-4af0-8ecc-0589eaf6a343","Type":"ContainerStarted","Data":"6a3f73f69cdeeb72610f85ee3dd31570878455d0dcbaa6aaef3926487412784a"} Jul 09 17:56:55.144834 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:55.144798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" event={"ID":"d49a149f-b69c-4af0-8ecc-0589eaf6a343","Type":"ContainerStarted","Data":"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6"} Jul 09 17:56:55.145214 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:55.144908 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:56:55.167590 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:56:55.167544 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" podStartSLOduration=1.7853808839999998 podStartE2EDuration="2.167528792s" totalImagesPullingTime="382.147908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:56:55.166338927 +0000 UTC m=+1757.615391356" watchObservedRunningTime="2026-07-09 17:56:55.167528792 +0000 UTC m=+1757.616581194" Jul 09 17:57:01.155358 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:01.155328 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:57:06.681792 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:06.681757 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:57:06.682168 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:06.681985 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" podUID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" containerName="maas-api" containerID="cri-o://b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6" gracePeriod=30 Jul 09 17:57:06.932657 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:06.932594 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:57:07.050737 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.050698 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls\") pod \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " Jul 09 17:57:07.050737 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.050739 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca\") pod \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " Jul 09 17:57:07.050977 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.050786 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf8zh\" (UniqueName: \"kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh\") pod \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\" (UID: \"d49a149f-b69c-4af0-8ecc-0589eaf6a343\") " Jul 09 17:57:07.051200 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.051167 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d49a149f-b69c-4af0-8ecc-0589eaf6a343" (UID: "d49a149f-b69c-4af0-8ecc-0589eaf6a343"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:57:07.052810 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.052772 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d49a149f-b69c-4af0-8ecc-0589eaf6a343" (UID: "d49a149f-b69c-4af0-8ecc-0589eaf6a343"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:57:07.052908 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.052855 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh" (OuterVolumeSpecName: "kube-api-access-nf8zh") pod "d49a149f-b69c-4af0-8ecc-0589eaf6a343" (UID: "d49a149f-b69c-4af0-8ecc-0589eaf6a343"). InnerVolumeSpecName "kube-api-access-nf8zh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:57:07.152058 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.152021 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d49a149f-b69c-4af0-8ecc-0589eaf6a343-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:57:07.152058 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.152051 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d49a149f-b69c-4af0-8ecc-0589eaf6a343-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:57:07.152058 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.152061 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf8zh\" (UniqueName: \"kubernetes.io/projected/d49a149f-b69c-4af0-8ecc-0589eaf6a343-kube-api-access-nf8zh\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:57:07.203307 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.203221 2577 generic.go:356] "Generic (PLEG): container finished" podID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" containerID="b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6" exitCode=0 Jul 09 17:57:07.203307 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.203264 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" Jul 09 17:57:07.203507 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.203316 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" event={"ID":"d49a149f-b69c-4af0-8ecc-0589eaf6a343","Type":"ContainerDied","Data":"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6"} Jul 09 17:57:07.203507 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.203387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg" event={"ID":"d49a149f-b69c-4af0-8ecc-0589eaf6a343","Type":"ContainerDied","Data":"6a3f73f69cdeeb72610f85ee3dd31570878455d0dcbaa6aaef3926487412784a"} Jul 09 17:57:07.203507 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.203416 2577 scope.go:117] "RemoveContainer" containerID="b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6" Jul 09 17:57:07.212928 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.212911 2577 scope.go:117] "RemoveContainer" containerID="b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6" Jul 09 17:57:07.213155 ip-10-0-130-12 kubenswrapper[2577]: E0709 17:57:07.213134 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6\": container with ID starting with b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6 not found: ID does not exist" containerID="b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6" Jul 09 17:57:07.213198 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.213164 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6"} err="failed to get container status \"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6\": rpc error: code = NotFound desc = could not find container \"b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6\": container with ID starting with b0abb5bac4f246ca63a0f1105532a29c0a77178418f7f4accf75563c103ccde6 not found: ID does not exist" Jul 09 17:57:07.232729 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.232699 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:57:07.239935 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:07.239905 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bc868f0-56f765b955-5s6dg"] Jul 09 17:57:08.075411 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:08.075379 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" path="/var/lib/kubelet/pods/d49a149f-b69c-4af0-8ecc-0589eaf6a343/volumes" Jul 09 17:57:37.732145 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.732103 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:57:37.732644 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.732629 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" containerName="maas-api" Jul 09 17:57:37.732686 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.732647 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" containerName="maas-api" Jul 09 17:57:37.732770 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.732759 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49a149f-b69c-4af0-8ecc-0589eaf6a343" containerName="maas-api" Jul 09 17:57:37.736280 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.736263 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.739052 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.739025 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-aaf858ec-serving-cert\"" Jul 09 17:57:37.765776 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.765750 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:57:37.814110 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.814079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8mhx\" (UniqueName: \"kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.814294 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.814129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.814294 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.814207 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.914980 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.914939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8mhx\" (UniqueName: \"kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.915160 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.914998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.915160 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.915039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.915680 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.915660 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.917487 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.917467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:37.930231 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:37.930202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8mhx\" (UniqueName: \"kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx\") pod \"maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:38.046652 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:38.046564 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:38.138331 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:38.135360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:57:38.141433 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:38.141382 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 17:57:38.415664 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:38.414094 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:57:38.419201 ip-10-0-130-12 kubenswrapper[2577]: W0709 17:57:38.419173 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ca1520_9dbf_4815_9784_ba01613fbb71.slice/crio-c6ac2b29b29bac0d2dbf900fc7cdea7c03ff01c6893ef7f4203b84c0228bbcad WatchSource:0}: Error finding container c6ac2b29b29bac0d2dbf900fc7cdea7c03ff01c6893ef7f4203b84c0228bbcad: Status 404 returned error can't find the container with id c6ac2b29b29bac0d2dbf900fc7cdea7c03ff01c6893ef7f4203b84c0228bbcad Jul 09 17:57:39.353445 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:39.353398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" event={"ID":"e2ca1520-9dbf-4815-9784-ba01613fbb71","Type":"ContainerStarted","Data":"d03d6c6126c19e37d671a21a7a160567df5559cfe5e772e44d81a203b5516fc3"} Jul 09 17:57:39.353938 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:39.353454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" event={"ID":"e2ca1520-9dbf-4815-9784-ba01613fbb71","Type":"ContainerStarted","Data":"c6ac2b29b29bac0d2dbf900fc7cdea7c03ff01c6893ef7f4203b84c0228bbcad"} Jul 09 17:57:39.353938 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:39.353535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:57:39.416894 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:39.416843 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" podStartSLOduration=2.015733495 podStartE2EDuration="2.416821659s" totalImagesPullingTime="401.088164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:57:39.416113077 +0000 UTC m=+1801.865165531" watchObservedRunningTime="2026-07-09 17:57:39.416821659 +0000 UTC m=+1801.865874060" Jul 09 17:57:45.364197 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:57:45.364164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:58:30.420923 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.420881 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:58:30.421507 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.421170 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" podUID="e2ca1520-9dbf-4815-9784-ba01613fbb71" containerName="maas-api" containerID="cri-o://d03d6c6126c19e37d671a21a7a160567df5559cfe5e772e44d81a203b5516fc3" gracePeriod=30 Jul 09 17:58:30.595090 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.595055 2577 generic.go:356] "Generic (PLEG): container finished" podID="e2ca1520-9dbf-4815-9784-ba01613fbb71" containerID="d03d6c6126c19e37d671a21a7a160567df5559cfe5e772e44d81a203b5516fc3" exitCode=0 Jul 09 17:58:30.595255 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.595101 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" event={"ID":"e2ca1520-9dbf-4815-9784-ba01613fbb71","Type":"ContainerDied","Data":"d03d6c6126c19e37d671a21a7a160567df5559cfe5e772e44d81a203b5516fc3"} Jul 09 17:58:30.668899 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.668874 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:58:30.781553 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.781462 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca\") pod \"e2ca1520-9dbf-4815-9784-ba01613fbb71\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " Jul 09 17:58:30.781712 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.781570 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8mhx\" (UniqueName: \"kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx\") pod \"e2ca1520-9dbf-4815-9784-ba01613fbb71\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " Jul 09 17:58:30.781712 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.781590 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls\") pod \"e2ca1520-9dbf-4815-9784-ba01613fbb71\" (UID: \"e2ca1520-9dbf-4815-9784-ba01613fbb71\") " Jul 09 17:58:30.781843 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.781818 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2ca1520-9dbf-4815-9784-ba01613fbb71" (UID: "e2ca1520-9dbf-4815-9784-ba01613fbb71"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:58:30.783717 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.783689 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2ca1520-9dbf-4815-9784-ba01613fbb71" (UID: "e2ca1520-9dbf-4815-9784-ba01613fbb71"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:58:30.783818 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.783738 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx" (OuterVolumeSpecName: "kube-api-access-t8mhx") pod "e2ca1520-9dbf-4815-9784-ba01613fbb71" (UID: "e2ca1520-9dbf-4815-9784-ba01613fbb71"). InnerVolumeSpecName "kube-api-access-t8mhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:58:30.882689 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.882653 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca1520-9dbf-4815-9784-ba01613fbb71-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:58:30.882689 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.882682 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8mhx\" (UniqueName: \"kubernetes.io/projected/e2ca1520-9dbf-4815-9784-ba01613fbb71-kube-api-access-t8mhx\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:58:30.882689 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:30.882693 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca1520-9dbf-4815-9784-ba01613fbb71-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 17:58:31.600360 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:31.600333 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" Jul 09 17:58:31.600360 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:31.600353 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml" event={"ID":"e2ca1520-9dbf-4815-9784-ba01613fbb71","Type":"ContainerDied","Data":"c6ac2b29b29bac0d2dbf900fc7cdea7c03ff01c6893ef7f4203b84c0228bbcad"} Jul 09 17:58:31.600840 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:31.600424 2577 scope.go:117] "RemoveContainer" containerID="d03d6c6126c19e37d671a21a7a160567df5559cfe5e772e44d81a203b5516fc3" Jul 09 17:58:31.624490 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:31.624463 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:58:31.634479 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:31.632216 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aaf858ec-7d8dfd56bd-pr4ml"] Jul 09 17:58:32.074215 ip-10-0-130-12 kubenswrapper[2577]: I0709 17:58:32.074184 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ca1520-9dbf-4815-9784-ba01613fbb71" path="/var/lib/kubelet/pods/e2ca1520-9dbf-4815-9784-ba01613fbb71/volumes" Jul 09 18:01:33.694967 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.694930 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:01:33.695451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.695341 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ca1520-9dbf-4815-9784-ba01613fbb71" containerName="maas-api" Jul 09 18:01:33.695451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.695356 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ca1520-9dbf-4815-9784-ba01613fbb71" containerName="maas-api" Jul 09 18:01:33.695451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.695438 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ca1520-9dbf-4815-9784-ba01613fbb71" containerName="maas-api" Jul 09 18:01:33.698501 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.698484 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.702075 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.702057 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2fa6b6-serving-cert\"" Jul 09 18:01:33.716897 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.716874 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:01:33.798069 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.798035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.798069 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.798076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndnsk\" (UniqueName: \"kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.798326 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.798126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.899451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.899415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.899639 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.899502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.899639 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.899529 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndnsk\" (UniqueName: \"kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.900286 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.900259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.901911 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.901889 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:33.910606 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:33.910583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndnsk\" (UniqueName: \"kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk\") pod \"maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:34.008741 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:34.008646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:34.148793 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:34.148762 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:01:34.150964 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:01:34.150938 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58896d1a_2e57_43a4_a5a3_ab9f0aa5a5c3.slice/crio-29a13bd7453467b6a4ed31a00bef780bb65b94cdf1895e39dd049841d0fe98bb WatchSource:0}: Error finding container 29a13bd7453467b6a4ed31a00bef780bb65b94cdf1895e39dd049841d0fe98bb: Status 404 returned error can't find the container with id 29a13bd7453467b6a4ed31a00bef780bb65b94cdf1895e39dd049841d0fe98bb Jul 09 18:01:34.152412 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:34.152390 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:01:34.455203 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:34.455166 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" event={"ID":"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3","Type":"ContainerStarted","Data":"29a13bd7453467b6a4ed31a00bef780bb65b94cdf1895e39dd049841d0fe98bb"} Jul 09 18:01:35.461590 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:35.461549 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" event={"ID":"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3","Type":"ContainerStarted","Data":"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0"} Jul 09 18:01:35.461996 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:35.461651 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:35.493629 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:35.493577 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" podStartSLOduration=1.3418301000000001 podStartE2EDuration="2.493556694s" totalImagesPullingTime="1.151726594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:01:35.492598245 +0000 UTC m=+2037.941650668" watchObservedRunningTime="2026-07-09 18:01:35.493556694 +0000 UTC m=+2037.942609095" Jul 09 18:01:41.471936 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.471907 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:01:41.703676 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.703642 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:01:41.707161 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.707143 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.722617 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.722564 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-712c34-serving-cert\"" Jul 09 18:01:41.742504 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.742468 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:01:41.767187 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.767154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.767339 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.767216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.767339 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.767280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdcx\" (UniqueName: \"kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.868150 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.868112 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.868335 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.868178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.868335 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.868219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdcx\" (UniqueName: \"kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.868335 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:01:41.868284 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-serving-cert: secret "maas-api-e2e-inf-b-712c34-serving-cert" not found Jul 09 18:01:41.868484 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:01:41.868355 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls podName:aedd684c-b121-41e7-95a9-69d8b684e047 nodeName:}" failed. No retries permitted until 2026-07-09 18:01:42.368336821 +0000 UTC m=+2044.817389205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls") pod "maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" (UID: "aedd684c-b121-41e7-95a9-69d8b684e047") : secret "maas-api-e2e-inf-b-712c34-serving-cert" not found Jul 09 18:01:41.869053 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.869031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:41.884442 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:41.884405 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdcx\" (UniqueName: \"kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:42.373373 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:42.373327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:42.375621 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:42.375598 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") pod \"maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:42.617499 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:42.617463 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:42.797012 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:42.796969 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:01:42.799381 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:01:42.799333 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedd684c_b121_41e7_95a9_69d8b684e047.slice/crio-432386e5e5ad0cd572146408b7b7d439cd2fab43123ed3de3e5087ad5a95704d WatchSource:0}: Error finding container 432386e5e5ad0cd572146408b7b7d439cd2fab43123ed3de3e5087ad5a95704d: Status 404 returned error can't find the container with id 432386e5e5ad0cd572146408b7b7d439cd2fab43123ed3de3e5087ad5a95704d Jul 09 18:01:43.501556 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:43.501454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" event={"ID":"aedd684c-b121-41e7-95a9-69d8b684e047","Type":"ContainerStarted","Data":"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535"} Jul 09 18:01:43.501556 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:43.501506 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" event={"ID":"aedd684c-b121-41e7-95a9-69d8b684e047","Type":"ContainerStarted","Data":"432386e5e5ad0cd572146408b7b7d439cd2fab43123ed3de3e5087ad5a95704d"} Jul 09 18:01:43.501781 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:43.501650 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:01:43.537854 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:43.537806 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" podStartSLOduration=2.163803297 podStartE2EDuration="2.53778784s" totalImagesPullingTime="373.984543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:01:43.535025304 +0000 UTC m=+2045.984077706" watchObservedRunningTime="2026-07-09 18:01:43.53778784 +0000 UTC m=+2045.986840242" Jul 09 18:01:49.512410 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:01:49.512354 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:02:38.166095 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:02:38.165945 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:02:38.172704 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:02:38.172682 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:03:14.085378 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.085331 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:14.088808 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.088791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.102200 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.102169 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:14.109194 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.109168 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-27f074-serving-cert\"" Jul 09 18:03:14.188605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.188558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.188790 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.188679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.188790 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.188709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gz5h\" (UniqueName: \"kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.289709 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.289669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.289709 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.289710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gz5h\" (UniqueName: \"kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.289941 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.289775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.290295 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.290271 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.292185 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.292162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.299629 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.299605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gz5h\" (UniqueName: \"kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h\") pod \"maas-api-e2e-reuse-27f074-68955d4566-zfwhs\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.399542 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.399447 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:14.545790 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.545764 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:14.547923 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:03:14.547889 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c9c700_f104_4965_8cce_beee7565dff2.slice/crio-fddf8d72669503ac65a5a6fae6d5b9b3b2d07ed63257b8d823c8b824950fc524 WatchSource:0}: Error finding container fddf8d72669503ac65a5a6fae6d5b9b3b2d07ed63257b8d823c8b824950fc524: Status 404 returned error can't find the container with id fddf8d72669503ac65a5a6fae6d5b9b3b2d07ed63257b8d823c8b824950fc524 Jul 09 18:03:14.931960 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:14.931931 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" event={"ID":"78c9c700-f104-4965-8cce-beee7565dff2","Type":"ContainerStarted","Data":"fddf8d72669503ac65a5a6fae6d5b9b3b2d07ed63257b8d823c8b824950fc524"} Jul 09 18:03:15.938236 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:15.938196 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" event={"ID":"78c9c700-f104-4965-8cce-beee7565dff2","Type":"ContainerStarted","Data":"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f"} Jul 09 18:03:15.938710 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:15.938309 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:15.978026 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:15.977965 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" podStartSLOduration=1.364029644 podStartE2EDuration="1.977947809s" totalImagesPullingTime="613.918165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:15.97359572 +0000 UTC m=+2138.422648123" watchObservedRunningTime="2026-07-09 18:03:15.977947809 +0000 UTC m=+2138.427000239" Jul 09 18:03:21.949361 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:21.949329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:39.769909 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:39.769875 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:39.770404 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:39.770072 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" podUID="78c9c700-f104-4965-8cce-beee7565dff2" containerName="maas-api" containerID="cri-o://11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f" gracePeriod=30 Jul 09 18:03:40.025213 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.025146 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:40.055117 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.055084 2577 generic.go:356] "Generic (PLEG): container finished" podID="78c9c700-f104-4965-8cce-beee7565dff2" containerID="11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f" exitCode=0 Jul 09 18:03:40.055258 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.055123 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" Jul 09 18:03:40.055258 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.055168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" event={"ID":"78c9c700-f104-4965-8cce-beee7565dff2","Type":"ContainerDied","Data":"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f"} Jul 09 18:03:40.055258 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.055214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs" event={"ID":"78c9c700-f104-4965-8cce-beee7565dff2","Type":"ContainerDied","Data":"fddf8d72669503ac65a5a6fae6d5b9b3b2d07ed63257b8d823c8b824950fc524"} Jul 09 18:03:40.055258 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.055235 2577 scope.go:117] "RemoveContainer" containerID="11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f" Jul 09 18:03:40.064304 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.064280 2577 scope.go:117] "RemoveContainer" containerID="11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f" Jul 09 18:03:40.064649 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:03:40.064618 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f\": container with ID starting with 11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f not found: ID does not exist" containerID="11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f" Jul 09 18:03:40.064742 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.064649 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f"} err="failed to get container status \"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f\": rpc error: code = NotFound desc = could not find container \"11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f\": container with ID starting with 11b0be5f07f5f61e2826abcee6504359237597a71a4a7973523bd9ce6a1afc6f not found: ID does not exist" Jul 09 18:03:40.117219 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.117183 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls\") pod \"78c9c700-f104-4965-8cce-beee7565dff2\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " Jul 09 18:03:40.117453 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.117260 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca\") pod \"78c9c700-f104-4965-8cce-beee7565dff2\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " Jul 09 18:03:40.117453 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.117302 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gz5h\" (UniqueName: \"kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h\") pod \"78c9c700-f104-4965-8cce-beee7565dff2\" (UID: \"78c9c700-f104-4965-8cce-beee7565dff2\") " Jul 09 18:03:40.117732 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.117707 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78c9c700-f104-4965-8cce-beee7565dff2" (UID: "78c9c700-f104-4965-8cce-beee7565dff2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:03:40.119563 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.119541 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h" (OuterVolumeSpecName: "kube-api-access-9gz5h") pod "78c9c700-f104-4965-8cce-beee7565dff2" (UID: "78c9c700-f104-4965-8cce-beee7565dff2"). InnerVolumeSpecName "kube-api-access-9gz5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:03:40.119645 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.119544 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78c9c700-f104-4965-8cce-beee7565dff2" (UID: "78c9c700-f104-4965-8cce-beee7565dff2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:03:40.218880 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.218846 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c9c700-f104-4965-8cce-beee7565dff2-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:03:40.218880 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.218879 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gz5h\" (UniqueName: \"kubernetes.io/projected/78c9c700-f104-4965-8cce-beee7565dff2-kube-api-access-9gz5h\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:03:40.219102 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.218894 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78c9c700-f104-4965-8cce-beee7565dff2-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:03:40.385174 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.385136 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:40.391437 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:40.391403 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-zfwhs"] Jul 09 18:03:42.074233 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:03:42.074195 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78c9c700-f104-4965-8cce-beee7565dff2" path="/var/lib/kubelet/pods/78c9c700-f104-4965-8cce-beee7565dff2/volumes" Jul 09 18:04:39.992223 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:39.992181 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:04:39.992801 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:39.992778 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78c9c700-f104-4965-8cce-beee7565dff2" containerName="maas-api" Jul 09 18:04:39.992863 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:39.992805 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="78c9c700-f104-4965-8cce-beee7565dff2" containerName="maas-api" Jul 09 18:04:39.992937 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:39.992922 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="78c9c700-f104-4965-8cce-beee7565dff2" containerName="maas-api" Jul 09 18:04:39.997612 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:39.997590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.002068 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.002044 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-27f074-serving-cert\"" Jul 09 18:04:40.015384 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.015337 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:04:40.144108 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.144068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.144294 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.144115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz676\" (UniqueName: \"kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.144294 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.144200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.245438 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.245335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.245596 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.245452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz676\" (UniqueName: \"kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.245596 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.245512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.246054 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.246031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.247936 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.247905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.255328 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.255304 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz676\" (UniqueName: \"kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676\") pod \"maas-api-e2e-reuse-27f074-68955d4566-ks5x8\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.310592 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.310549 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:40.469797 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:40.469773 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:04:40.471993 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:04:40.471965 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d2e3a1_e7ef_4d2f_a63e_3edbc96d77f2.slice/crio-5c6f37d72c5f8ec658dc4ec4d027315ca989a6e17ccc5fde068be82094edbb4a WatchSource:0}: Error finding container 5c6f37d72c5f8ec658dc4ec4d027315ca989a6e17ccc5fde068be82094edbb4a: Status 404 returned error can't find the container with id 5c6f37d72c5f8ec658dc4ec4d027315ca989a6e17ccc5fde068be82094edbb4a Jul 09 18:04:41.365274 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:41.365181 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" event={"ID":"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2","Type":"ContainerStarted","Data":"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e"} Jul 09 18:04:41.365274 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:41.365232 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" event={"ID":"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2","Type":"ContainerStarted","Data":"5c6f37d72c5f8ec658dc4ec4d027315ca989a6e17ccc5fde068be82094edbb4a"} Jul 09 18:04:41.365697 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:41.365289 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:04:41.393609 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:41.393558 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" podStartSLOduration=1.758998931 podStartE2EDuration="2.39353813s" totalImagesPullingTime="634.539199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:41.393337342 +0000 UTC m=+2223.842389744" watchObservedRunningTime="2026-07-09 18:04:41.39353813 +0000 UTC m=+2223.842590561" Jul 09 18:04:47.376068 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:04:47.376036 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:05:26.511333 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.511252 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:05:26.511873 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.511506 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" podUID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" containerName="maas-api" containerID="cri-o://afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e" gracePeriod=30 Jul 09 18:05:26.762022 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.761962 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:05:26.862078 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.862037 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls\") pod \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " Jul 09 18:05:26.862270 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.862099 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz676\" (UniqueName: \"kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676\") pod \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " Jul 09 18:05:26.862270 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.862135 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca\") pod \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\" (UID: \"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2\") " Jul 09 18:05:26.862601 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.862575 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" (UID: "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:05:26.864194 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.864161 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" (UID: "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:05:26.864283 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.864202 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676" (OuterVolumeSpecName: "kube-api-access-dz676") pod "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" (UID: "78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2"). InnerVolumeSpecName "kube-api-access-dz676". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:05:26.963018 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.962976 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:05:26.963018 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.963012 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz676\" (UniqueName: \"kubernetes.io/projected/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-kube-api-access-dz676\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:05:26.963018 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:26.963022 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:05:27.592081 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.592049 2577 generic.go:356] "Generic (PLEG): container finished" podID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" containerID="afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e" exitCode=0 Jul 09 18:05:27.592572 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.592096 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" Jul 09 18:05:27.592572 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.592130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" event={"ID":"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2","Type":"ContainerDied","Data":"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e"} Jul 09 18:05:27.592572 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.592182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8" event={"ID":"78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2","Type":"ContainerDied","Data":"5c6f37d72c5f8ec658dc4ec4d027315ca989a6e17ccc5fde068be82094edbb4a"} Jul 09 18:05:27.592572 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.592204 2577 scope.go:117] "RemoveContainer" containerID="afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e" Jul 09 18:05:27.601652 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.601631 2577 scope.go:117] "RemoveContainer" containerID="afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e" Jul 09 18:05:27.601929 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:05:27.601911 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e\": container with ID starting with afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e not found: ID does not exist" containerID="afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e" Jul 09 18:05:27.601997 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.601937 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e"} err="failed to get container status \"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e\": rpc error: code = NotFound desc = could not find container \"afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e\": container with ID starting with afad53cabcb1b456223113f662fc8b6a2fb6457dbb606ade853a7acd31e8ca8e not found: ID does not exist" Jul 09 18:05:27.613481 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.613455 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:05:27.618388 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:27.618351 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-27f074-68955d4566-ks5x8"] Jul 09 18:05:28.074999 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:05:28.074970 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" path="/var/lib/kubelet/pods/78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2/volumes" Jul 09 18:06:32.954948 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:32.953977 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:06:32.954948 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:32.954316 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" podUID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" containerName="maas-api" containerID="cri-o://21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0" gracePeriod=30 Jul 09 18:06:33.206348 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.206280 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:06:33.358211 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.358173 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndnsk\" (UniqueName: \"kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk\") pod \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " Jul 09 18:06:33.358414 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.358267 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca\") pod \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " Jul 09 18:06:33.358414 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.358318 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls\") pod \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\" (UID: \"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3\") " Jul 09 18:06:33.358749 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.358722 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" (UID: "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:33.360442 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.360422 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk" (OuterVolumeSpecName: "kube-api-access-ndnsk") pod "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" (UID: "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3"). InnerVolumeSpecName "kube-api-access-ndnsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:06:33.360555 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.360439 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" (UID: "58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.459271 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.459167 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.459271 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.459212 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.459271 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.459223 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndnsk\" (UniqueName: \"kubernetes.io/projected/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3-kube-api-access-ndnsk\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.916209 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.916174 2577 generic.go:356] "Generic (PLEG): container finished" podID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" containerID="21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0" exitCode=0 Jul 09 18:06:33.916209 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.916214 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" Jul 09 18:06:33.916451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.916214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" event={"ID":"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3","Type":"ContainerDied","Data":"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0"} Jul 09 18:06:33.916451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.916354 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp" event={"ID":"58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3","Type":"ContainerDied","Data":"29a13bd7453467b6a4ed31a00bef780bb65b94cdf1895e39dd049841d0fe98bb"} Jul 09 18:06:33.916451 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.916419 2577 scope.go:117] "RemoveContainer" containerID="21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0" Jul 09 18:06:33.925737 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.925717 2577 scope.go:117] "RemoveContainer" containerID="21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0" Jul 09 18:06:33.926009 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:06:33.925986 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0\": container with ID starting with 21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0 not found: ID does not exist" containerID="21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0" Jul 09 18:06:33.926061 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.926021 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0"} err="failed to get container status \"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0\": rpc error: code = NotFound desc = could not find container \"21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0\": container with ID starting with 21c5ba12b5f87c17ccaad99d5266e9a3fc2f02163d1737bfe2891db636d0e0f0 not found: ID does not exist" Jul 09 18:06:33.936943 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.936891 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:06:33.938734 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:33.938711 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2fa6b6-644d456f88-7wlqp"] Jul 09 18:06:34.074268 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:06:34.074234 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" path="/var/lib/kubelet/pods/58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3/volumes" Jul 09 18:07:15.362789 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.362750 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:07:15.363421 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.363039 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" podUID="aedd684c-b121-41e7-95a9-69d8b684e047" containerName="maas-api" containerID="cri-o://42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535" gracePeriod=30 Jul 09 18:07:15.619749 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.619685 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:07:15.737230 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.737189 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sdcx\" (UniqueName: \"kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx\") pod \"aedd684c-b121-41e7-95a9-69d8b684e047\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " Jul 09 18:07:15.737444 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.737264 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca\") pod \"aedd684c-b121-41e7-95a9-69d8b684e047\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " Jul 09 18:07:15.737498 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.737455 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") pod \"aedd684c-b121-41e7-95a9-69d8b684e047\" (UID: \"aedd684c-b121-41e7-95a9-69d8b684e047\") " Jul 09 18:07:15.737679 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.737654 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aedd684c-b121-41e7-95a9-69d8b684e047" (UID: "aedd684c-b121-41e7-95a9-69d8b684e047"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:07:15.737826 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.737812 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedd684c-b121-41e7-95a9-69d8b684e047-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:15.739481 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.739460 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aedd684c-b121-41e7-95a9-69d8b684e047" (UID: "aedd684c-b121-41e7-95a9-69d8b684e047"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:07:15.739481 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.739472 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx" (OuterVolumeSpecName: "kube-api-access-2sdcx") pod "aedd684c-b121-41e7-95a9-69d8b684e047" (UID: "aedd684c-b121-41e7-95a9-69d8b684e047"). InnerVolumeSpecName "kube-api-access-2sdcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:07:15.838705 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.838672 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sdcx\" (UniqueName: \"kubernetes.io/projected/aedd684c-b121-41e7-95a9-69d8b684e047-kube-api-access-2sdcx\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:15.838705 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:15.838700 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedd684c-b121-41e7-95a9-69d8b684e047-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:16.115418 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.115360 2577 generic.go:356] "Generic (PLEG): container finished" podID="aedd684c-b121-41e7-95a9-69d8b684e047" containerID="42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535" exitCode=0 Jul 09 18:07:16.115618 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.115422 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" Jul 09 18:07:16.115618 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.115436 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" event={"ID":"aedd684c-b121-41e7-95a9-69d8b684e047","Type":"ContainerDied","Data":"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535"} Jul 09 18:07:16.115618 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.115489 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm" event={"ID":"aedd684c-b121-41e7-95a9-69d8b684e047","Type":"ContainerDied","Data":"432386e5e5ad0cd572146408b7b7d439cd2fab43123ed3de3e5087ad5a95704d"} Jul 09 18:07:16.115618 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.115512 2577 scope.go:117] "RemoveContainer" containerID="42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535" Jul 09 18:07:16.124542 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.124525 2577 scope.go:117] "RemoveContainer" containerID="42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535" Jul 09 18:07:16.124802 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:16.124782 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535\": container with ID starting with 42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535 not found: ID does not exist" containerID="42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535" Jul 09 18:07:16.124868 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.124814 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535"} err="failed to get container status \"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535\": rpc error: code = NotFound desc = could not find container \"42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535\": container with ID starting with 42ff726956698696bd43063f26574a4312763c97c3a7795167aad6f26ee83535 not found: ID does not exist" Jul 09 18:07:16.132878 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.132850 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:07:16.136776 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:16.136756 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-712c34-6d95b9b68-4vzxm"] Jul 09 18:07:18.075291 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:18.075258 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aedd684c-b121-41e7-95a9-69d8b684e047" path="/var/lib/kubelet/pods/aedd684c-b121-41e7-95a9-69d8b684e047/volumes" Jul 09 18:07:25.215360 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215317 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:25.215898 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215870 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aedd684c-b121-41e7-95a9-69d8b684e047" containerName="maas-api" Jul 09 18:07:25.215898 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215886 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aedd684c-b121-41e7-95a9-69d8b684e047" containerName="maas-api" Jul 09 18:07:25.215989 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215907 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" containerName="maas-api" Jul 09 18:07:25.215989 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215916 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" containerName="maas-api" Jul 09 18:07:25.215989 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215936 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" containerName="maas-api" Jul 09 18:07:25.215989 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.215946 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" containerName="maas-api" Jul 09 18:07:25.216136 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.216047 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aedd684c-b121-41e7-95a9-69d8b684e047" containerName="maas-api" Jul 09 18:07:25.216136 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.216059 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d2e3a1-e7ef-4d2f-a63e-3edbc96d77f2" containerName="maas-api" Jul 09 18:07:25.216136 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.216071 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="58896d1a-2e57-43a4-a5a3-ab9f0aa5a5c3" containerName="maas-api" Jul 09 18:07:25.220717 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.220688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.224802 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.224777 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fe7a7c-serving-cert\"" Jul 09 18:07:25.232789 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.232760 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:25.331201 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.331158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2nzp\" (UniqueName: \"kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.331413 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.331210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.331413 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.331344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.432618 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.432575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.432792 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.432648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2nzp\" (UniqueName: \"kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.432792 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.432682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.432869 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:25.432844 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-serving-cert: secret "maas-api-e2e-api-a-fe7a7c-serving-cert" not found Jul 09 18:07:25.432920 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:25.432910 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls podName:744b6a9e-5c11-4d9d-998e-384e072fe906 nodeName:}" failed. No retries permitted until 2026-07-09 18:07:25.932889028 +0000 UTC m=+2388.381941407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls") pod "maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" (UID: "744b6a9e-5c11-4d9d-998e-384e072fe906") : secret "maas-api-e2e-api-a-fe7a7c-serving-cert" not found Jul 09 18:07:25.433300 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.433280 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.441564 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.441531 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2nzp\" (UniqueName: \"kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.938964 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.938923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:25.941562 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:25.941530 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") pod \"maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:26.132959 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:26.132922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:26.290600 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:26.290572 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:26.292424 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:07:26.292390 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744b6a9e_5c11_4d9d_998e_384e072fe906.slice/crio-cd63c80b36b4a40a3823b924b8a24bb3a89bc4b1e670467b1f65ab7ad79b10f3 WatchSource:0}: Error finding container cd63c80b36b4a40a3823b924b8a24bb3a89bc4b1e670467b1f65ab7ad79b10f3: Status 404 returned error can't find the container with id cd63c80b36b4a40a3823b924b8a24bb3a89bc4b1e670467b1f65ab7ad79b10f3 Jul 09 18:07:26.293711 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:26.293692 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:07:27.169773 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:27.169730 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" event={"ID":"744b6a9e-5c11-4d9d-998e-384e072fe906","Type":"ContainerStarted","Data":"cd63c80b36b4a40a3823b924b8a24bb3a89bc4b1e670467b1f65ab7ad79b10f3"} Jul 09 18:07:28.180163 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:28.180062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" event={"ID":"744b6a9e-5c11-4d9d-998e-384e072fe906","Type":"ContainerStarted","Data":"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90"} Jul 09 18:07:28.180589 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:28.180206 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:33.265844 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.265793 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" podStartSLOduration=6.634029469 podStartE2EDuration="8.26577377s" totalImagesPullingTime="1.631744301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:07:28.201770609 +0000 UTC m=+2390.650823011" watchObservedRunningTime="2026-07-09 18:07:33.26577377 +0000 UTC m=+2395.714826227" Jul 09 18:07:33.266460 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.266441 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:07:33.270132 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.270109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.272731 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.272708 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-589edb-serving-cert\"" Jul 09 18:07:33.290286 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.290255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:07:33.301684 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.301651 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.301855 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.301690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcqfv\" (UniqueName: \"kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.301855 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.301816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.403045 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.403011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.403045 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.403047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcqfv\" (UniqueName: \"kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.403290 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.403180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.403394 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:33.403346 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-serving-cert: secret "maas-api-e2e-api-b-589edb-serving-cert" not found Jul 09 18:07:33.403470 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:33.403456 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls podName:f168033f-98a6-401e-9770-6d9e463d779f nodeName:}" failed. No retries permitted until 2026-07-09 18:07:33.903432613 +0000 UTC m=+2396.352485002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls") pod "maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" (UID: "f168033f-98a6-401e-9770-6d9e463d779f") : secret "maas-api-e2e-api-b-589edb-serving-cert" not found Jul 09 18:07:33.403871 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.403849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.426970 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.426932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcqfv\" (UniqueName: \"kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.908159 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.908118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:33.910698 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:33.910670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") pod \"maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:34.190919 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:34.190843 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:34.194354 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:34.194322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:34.341425 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:34.341315 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:07:34.343882 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:07:34.343850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf168033f_98a6_401e_9770_6d9e463d779f.slice/crio-e777d074b8db166fa067a42a25e95851f6e58a876f7a2234e0e34c7fa283b65d WatchSource:0}: Error finding container e777d074b8db166fa067a42a25e95851f6e58a876f7a2234e0e34c7fa283b65d: Status 404 returned error can't find the container with id e777d074b8db166fa067a42a25e95851f6e58a876f7a2234e0e34c7fa283b65d Jul 09 18:07:35.215261 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:35.215229 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" event={"ID":"f168033f-98a6-401e-9770-6d9e463d779f","Type":"ContainerStarted","Data":"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e"} Jul 09 18:07:35.215471 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:35.215290 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" event={"ID":"f168033f-98a6-401e-9770-6d9e463d779f","Type":"ContainerStarted","Data":"e777d074b8db166fa067a42a25e95851f6e58a876f7a2234e0e34c7fa283b65d"} Jul 09 18:07:35.215471 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:35.215317 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:38.194709 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:38.194545 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:07:38.208428 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:38.202440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:07:41.225535 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:41.225502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:07:41.243146 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:41.243092 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" podStartSLOduration=7.941730665 podStartE2EDuration="8.243075883s" totalImagesPullingTime="301.345218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:07:35.234355018 +0000 UTC m=+2397.683407421" watchObservedRunningTime="2026-07-09 18:07:41.243075883 +0000 UTC m=+2403.692128285" Jul 09 18:07:46.634452 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:46.634415 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:46.635272 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:46.635208 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" podUID="744b6a9e-5c11-4d9d-998e-384e072fe906" containerName="maas-api" containerID="cri-o://f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90" gracePeriod=30 Jul 09 18:07:46.891708 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:46.891643 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:47.028895 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.028853 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca\") pod \"744b6a9e-5c11-4d9d-998e-384e072fe906\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " Jul 09 18:07:47.029072 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.028937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2nzp\" (UniqueName: \"kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp\") pod \"744b6a9e-5c11-4d9d-998e-384e072fe906\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " Jul 09 18:07:47.029072 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.028995 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") pod \"744b6a9e-5c11-4d9d-998e-384e072fe906\" (UID: \"744b6a9e-5c11-4d9d-998e-384e072fe906\") " Jul 09 18:07:47.029250 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.029223 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "744b6a9e-5c11-4d9d-998e-384e072fe906" (UID: "744b6a9e-5c11-4d9d-998e-384e072fe906"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:07:47.031075 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.031050 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "744b6a9e-5c11-4d9d-998e-384e072fe906" (UID: "744b6a9e-5c11-4d9d-998e-384e072fe906"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:07:47.031201 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.031117 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp" (OuterVolumeSpecName: "kube-api-access-l2nzp") pod "744b6a9e-5c11-4d9d-998e-384e072fe906" (UID: "744b6a9e-5c11-4d9d-998e-384e072fe906"). InnerVolumeSpecName "kube-api-access-l2nzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:07:47.130325 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.130294 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/744b6a9e-5c11-4d9d-998e-384e072fe906-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:47.130325 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.130319 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2nzp\" (UniqueName: \"kubernetes.io/projected/744b6a9e-5c11-4d9d-998e-384e072fe906-kube-api-access-l2nzp\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:47.130325 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.130328 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/744b6a9e-5c11-4d9d-998e-384e072fe906-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:07:47.273489 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.273389 2577 generic.go:356] "Generic (PLEG): container finished" podID="744b6a9e-5c11-4d9d-998e-384e072fe906" containerID="f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90" exitCode=0 Jul 09 18:07:47.273489 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.273432 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" Jul 09 18:07:47.273489 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.273457 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" event={"ID":"744b6a9e-5c11-4d9d-998e-384e072fe906","Type":"ContainerDied","Data":"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90"} Jul 09 18:07:47.273774 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.273517 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf" event={"ID":"744b6a9e-5c11-4d9d-998e-384e072fe906","Type":"ContainerDied","Data":"cd63c80b36b4a40a3823b924b8a24bb3a89bc4b1e670467b1f65ab7ad79b10f3"} Jul 09 18:07:47.273774 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.273548 2577 scope.go:117] "RemoveContainer" containerID="f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90" Jul 09 18:07:47.282449 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.282429 2577 scope.go:117] "RemoveContainer" containerID="f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90" Jul 09 18:07:47.282721 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:07:47.282699 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90\": container with ID starting with f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90 not found: ID does not exist" containerID="f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90" Jul 09 18:07:47.282776 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.282735 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90"} err="failed to get container status \"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90\": rpc error: code = NotFound desc = could not find container \"f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90\": container with ID starting with f6cb70ee62fea128baef5b4bba1943f23c3557105a3e6bd79fd1fe2900a7fa90 not found: ID does not exist" Jul 09 18:07:47.309583 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.309554 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:47.311105 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:47.311087 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fe7a7c-84f45f9495-d6bhf"] Jul 09 18:07:48.074831 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:07:48.074797 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="744b6a9e-5c11-4d9d-998e-384e072fe906" path="/var/lib/kubelet/pods/744b6a9e-5c11-4d9d-998e-384e072fe906/volumes" Jul 09 18:08:03.496806 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.496541 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:08:03.497330 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.496826 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" podUID="f168033f-98a6-401e-9770-6d9e463d779f" containerName="maas-api" containerID="cri-o://3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e" gracePeriod=30 Jul 09 18:08:03.749393 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.749312 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:08:03.869157 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.869120 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcqfv\" (UniqueName: \"kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv\") pod \"f168033f-98a6-401e-9770-6d9e463d779f\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " Jul 09 18:08:03.869157 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.869163 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") pod \"f168033f-98a6-401e-9770-6d9e463d779f\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " Jul 09 18:08:03.869407 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.869197 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca\") pod \"f168033f-98a6-401e-9770-6d9e463d779f\" (UID: \"f168033f-98a6-401e-9770-6d9e463d779f\") " Jul 09 18:08:03.869587 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.869565 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f168033f-98a6-401e-9770-6d9e463d779f" (UID: "f168033f-98a6-401e-9770-6d9e463d779f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:08:03.871291 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.871262 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f168033f-98a6-401e-9770-6d9e463d779f" (UID: "f168033f-98a6-401e-9770-6d9e463d779f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:08:03.871413 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.871304 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv" (OuterVolumeSpecName: "kube-api-access-gcqfv") pod "f168033f-98a6-401e-9770-6d9e463d779f" (UID: "f168033f-98a6-401e-9770-6d9e463d779f"). InnerVolumeSpecName "kube-api-access-gcqfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:08:03.970779 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.970741 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f168033f-98a6-401e-9770-6d9e463d779f-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:08:03.970779 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.970769 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcqfv\" (UniqueName: \"kubernetes.io/projected/f168033f-98a6-401e-9770-6d9e463d779f-kube-api-access-gcqfv\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:08:03.970779 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:03.970780 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f168033f-98a6-401e-9770-6d9e463d779f-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:08:04.353573 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.353490 2577 generic.go:356] "Generic (PLEG): container finished" podID="f168033f-98a6-401e-9770-6d9e463d779f" containerID="3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e" exitCode=0 Jul 09 18:08:04.353573 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.353534 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" Jul 09 18:08:04.353786 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.353537 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" event={"ID":"f168033f-98a6-401e-9770-6d9e463d779f","Type":"ContainerDied","Data":"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e"} Jul 09 18:08:04.353786 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.353648 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs" event={"ID":"f168033f-98a6-401e-9770-6d9e463d779f","Type":"ContainerDied","Data":"e777d074b8db166fa067a42a25e95851f6e58a876f7a2234e0e34c7fa283b65d"} Jul 09 18:08:04.353786 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.353671 2577 scope.go:117] "RemoveContainer" containerID="3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e" Jul 09 18:08:04.362338 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.362320 2577 scope.go:117] "RemoveContainer" containerID="3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e" Jul 09 18:08:04.362628 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:08:04.362609 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e\": container with ID starting with 3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e not found: ID does not exist" containerID="3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e" Jul 09 18:08:04.362677 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.362640 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e"} err="failed to get container status \"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e\": rpc error: code = NotFound desc = could not find container \"3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e\": container with ID starting with 3a624df026867bb65e320c8910ece52f7954b3a1a27f2771175d288634b2594e not found: ID does not exist" Jul 09 18:08:04.372602 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.372572 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:08:04.378427 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:04.378405 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-589edb-d8dbfff84-8nvxs"] Jul 09 18:08:06.074429 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:06.074391 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f168033f-98a6-401e-9770-6d9e463d779f" path="/var/lib/kubelet/pods/f168033f-98a6-401e-9770-6d9e463d779f/volumes" Jul 09 18:08:13.634131 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634098 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:08:13.634662 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634565 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="744b6a9e-5c11-4d9d-998e-384e072fe906" containerName="maas-api" Jul 09 18:08:13.634662 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634584 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="744b6a9e-5c11-4d9d-998e-384e072fe906" containerName="maas-api" Jul 09 18:08:13.634662 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634631 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f168033f-98a6-401e-9770-6d9e463d779f" containerName="maas-api" Jul 09 18:08:13.634662 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634639 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f168033f-98a6-401e-9770-6d9e463d779f" containerName="maas-api" Jul 09 18:08:13.634791 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634757 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f168033f-98a6-401e-9770-6d9e463d779f" containerName="maas-api" Jul 09 18:08:13.634791 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.634771 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="744b6a9e-5c11-4d9d-998e-384e072fe906" containerName="maas-api" Jul 09 18:08:13.639354 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.639333 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.641549 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.641520 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-09d279-serving-cert\"" Jul 09 18:08:13.648454 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.648430 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:08:13.754631 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.754584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.754812 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.754672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfrm\" (UniqueName: \"kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.754812 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.754698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.855866 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.855831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfrm\" (UniqueName: \"kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.855866 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.855869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.856073 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.855917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.856073 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:08:13.856062 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-serving-cert: secret "maas-api-e2e-shared-a-09d279-serving-cert" not found Jul 09 18:08:13.856143 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:08:13.856128 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls podName:ef73c41c-cade-4976-a669-11b67b9baf66 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:14.356111394 +0000 UTC m=+2436.805163773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls") pod "maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" (UID: "ef73c41c-cade-4976-a669-11b67b9baf66") : secret "maas-api-e2e-shared-a-09d279-serving-cert" not found Jul 09 18:08:13.856623 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.856606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:13.868530 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:13.868503 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfrm\" (UniqueName: \"kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:14.360834 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:14.360796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:14.363205 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:14.363174 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") pod \"maas-api-e2e-shared-a-09d279-77985cf94-8jz4t\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:14.551840 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:14.551808 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:14.696321 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:14.696299 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:08:14.698514 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:08:14.698484 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef73c41c_cade_4976_a669_11b67b9baf66.slice/crio-807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79 WatchSource:0}: Error finding container 807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79: Status 404 returned error can't find the container with id 807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79 Jul 09 18:08:15.409409 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:15.409351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" event={"ID":"ef73c41c-cade-4976-a669-11b67b9baf66","Type":"ContainerStarted","Data":"c18d075f398fed66d0caf9ffea27b6a852a359adec66022376d58bab1d3b773a"} Jul 09 18:08:15.409597 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:15.409427 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" event={"ID":"ef73c41c-cade-4976-a669-11b67b9baf66","Type":"ContainerStarted","Data":"807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79"} Jul 09 18:08:15.409597 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:15.409557 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:21.419382 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.419333 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:08:21.438090 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.438038 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" podStartSLOduration=8.096241453 podStartE2EDuration="8.438020944s" totalImagesPullingTime="341.779491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:15.471161961 +0000 UTC m=+2437.920214360" watchObservedRunningTime="2026-07-09 18:08:21.438020944 +0000 UTC m=+2443.887073345" Jul 09 18:08:21.598436 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.598393 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:08:21.602483 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.602460 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.605186 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.605161 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e0b9f9-serving-cert\"" Jul 09 18:08:21.614862 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.614836 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:08:21.619568 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.619542 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn4wr\" (UniqueName: \"kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.619666 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.619581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.619715 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.619673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.720129 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.720032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.720129 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.720092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn4wr\" (UniqueName: \"kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.720129 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.720118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.720425 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:08:21.720172 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-serving-cert: secret "maas-api-e2e-shared-b-e0b9f9-serving-cert" not found Jul 09 18:08:21.720425 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:08:21.720233 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls podName:ad1ba2bb-d444-4e1a-b618-39b92ab7be99 nodeName:}" failed. No retries permitted until 2026-07-09 18:08:22.220217792 +0000 UTC m=+2444.669270172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls") pod "maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" (UID: "ad1ba2bb-d444-4e1a-b618-39b92ab7be99") : secret "maas-api-e2e-shared-b-e0b9f9-serving-cert" not found Jul 09 18:08:21.720760 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.720741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:21.729286 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:21.729266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn4wr\" (UniqueName: \"kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:22.224043 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:22.224009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:22.226605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:22.226572 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:22.516167 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:22.516083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:22.652923 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:22.652898 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:08:22.655654 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:08:22.655625 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad1ba2bb_d444_4e1a_b618_39b92ab7be99.slice/crio-72d48591ab7247cb6c6c1223416ec4cf1b65e8134362c56a7889a21fcc07cd64 WatchSource:0}: Error finding container 72d48591ab7247cb6c6c1223416ec4cf1b65e8134362c56a7889a21fcc07cd64: Status 404 returned error can't find the container with id 72d48591ab7247cb6c6c1223416ec4cf1b65e8134362c56a7889a21fcc07cd64 Jul 09 18:08:23.449093 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:23.449066 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" event={"ID":"ad1ba2bb-d444-4e1a-b618-39b92ab7be99","Type":"ContainerStarted","Data":"72d48591ab7247cb6c6c1223416ec4cf1b65e8134362c56a7889a21fcc07cd64"} Jul 09 18:08:24.455586 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:24.455551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" event={"ID":"ad1ba2bb-d444-4e1a-b618-39b92ab7be99","Type":"ContainerStarted","Data":"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582"} Jul 09 18:08:24.456009 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:24.455638 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:08:24.472517 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:24.472454 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" podStartSLOduration=2.699027881 podStartE2EDuration="3.472428136s" totalImagesPullingTime="773.400255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:08:24.472136812 +0000 UTC m=+2446.921189216" watchObservedRunningTime="2026-07-09 18:08:24.472428136 +0000 UTC m=+2446.921480538" Jul 09 18:08:30.465579 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:08:30.465543 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:12:38.222978 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:12:38.222867 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:12:38.238075 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:12:38.238054 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:15:32.223004 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.222904 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:15:32.223599 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.223136 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" podUID="ef73c41c-cade-4976-a669-11b67b9baf66" containerName="maas-api" containerID="cri-o://c18d075f398fed66d0caf9ffea27b6a852a359adec66022376d58bab1d3b773a" gracePeriod=30 Jul 09 18:15:32.474423 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.474338 2577 generic.go:356] "Generic (PLEG): container finished" podID="ef73c41c-cade-4976-a669-11b67b9baf66" containerID="c18d075f398fed66d0caf9ffea27b6a852a359adec66022376d58bab1d3b773a" exitCode=0 Jul 09 18:15:32.474423 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.474409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" event={"ID":"ef73c41c-cade-4976-a669-11b67b9baf66","Type":"ContainerDied","Data":"c18d075f398fed66d0caf9ffea27b6a852a359adec66022376d58bab1d3b773a"} Jul 09 18:15:32.474579 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.474464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" event={"ID":"ef73c41c-cade-4976-a669-11b67b9baf66","Type":"ContainerDied","Data":"807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79"} Jul 09 18:15:32.474579 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.474482 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="807cfb07ce39bc2ba1c9a835d6f11c543fab57e9ae694be1afa704f91fa77d79" Jul 09 18:15:32.474678 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.474663 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:15:32.575013 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.574975 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqfrm\" (UniqueName: \"kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm\") pod \"ef73c41c-cade-4976-a669-11b67b9baf66\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " Jul 09 18:15:32.575193 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.575039 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") pod \"ef73c41c-cade-4976-a669-11b67b9baf66\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " Jul 09 18:15:32.575193 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.575120 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca\") pod \"ef73c41c-cade-4976-a669-11b67b9baf66\" (UID: \"ef73c41c-cade-4976-a669-11b67b9baf66\") " Jul 09 18:15:32.575519 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.575485 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef73c41c-cade-4976-a669-11b67b9baf66" (UID: "ef73c41c-cade-4976-a669-11b67b9baf66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:15:32.577215 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.577190 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef73c41c-cade-4976-a669-11b67b9baf66" (UID: "ef73c41c-cade-4976-a669-11b67b9baf66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:15:32.577314 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.577230 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm" (OuterVolumeSpecName: "kube-api-access-dqfrm") pod "ef73c41c-cade-4976-a669-11b67b9baf66" (UID: "ef73c41c-cade-4976-a669-11b67b9baf66"). InnerVolumeSpecName "kube-api-access-dqfrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:15:32.675961 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.675924 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef73c41c-cade-4976-a669-11b67b9baf66-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:32.675961 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.675954 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqfrm\" (UniqueName: \"kubernetes.io/projected/ef73c41c-cade-4976-a669-11b67b9baf66-kube-api-access-dqfrm\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:32.675961 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:32.675964 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef73c41c-cade-4976-a669-11b67b9baf66-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:33.478926 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:33.478891 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t" Jul 09 18:15:33.499157 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:33.499127 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:15:33.502504 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:33.502481 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-09d279-77985cf94-8jz4t"] Jul 09 18:15:34.073818 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:34.073784 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef73c41c-cade-4976-a669-11b67b9baf66" path="/var/lib/kubelet/pods/ef73c41c-cade-4976-a669-11b67b9baf66/volumes" Jul 09 18:15:38.340759 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:38.340726 2577 scope.go:117] "RemoveContainer" containerID="c18d075f398fed66d0caf9ffea27b6a852a359adec66022376d58bab1d3b773a" Jul 09 18:15:39.445037 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.445000 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:15:39.445536 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.445275 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" podUID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" containerName="maas-api" containerID="cri-o://6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582" gracePeriod=30 Jul 09 18:15:39.686204 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.686175 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:15:39.732787 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.732700 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca\") pod \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " Jul 09 18:15:39.732787 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.732751 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn4wr\" (UniqueName: \"kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr\") pod \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " Jul 09 18:15:39.732998 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.732827 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") pod \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\" (UID: \"ad1ba2bb-d444-4e1a-b618-39b92ab7be99\") " Jul 09 18:15:39.733077 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.733047 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad1ba2bb-d444-4e1a-b618-39b92ab7be99" (UID: "ad1ba2bb-d444-4e1a-b618-39b92ab7be99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:15:39.734841 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.734813 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr" (OuterVolumeSpecName: "kube-api-access-bn4wr") pod "ad1ba2bb-d444-4e1a-b618-39b92ab7be99" (UID: "ad1ba2bb-d444-4e1a-b618-39b92ab7be99"). InnerVolumeSpecName "kube-api-access-bn4wr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:15:39.734971 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.734924 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad1ba2bb-d444-4e1a-b618-39b92ab7be99" (UID: "ad1ba2bb-d444-4e1a-b618-39b92ab7be99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:15:39.834406 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.834348 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-openshift-service-ca\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:39.834406 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.834400 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn4wr\" (UniqueName: \"kubernetes.io/projected/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-kube-api-access-bn4wr\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:39.834406 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:39.834413 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad1ba2bb-d444-4e1a-b618-39b92ab7be99-maas-api-tls\") on node \"ip-10-0-130-12.ec2.internal\" DevicePath \"\"" Jul 09 18:15:40.512014 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.511931 2577 generic.go:356] "Generic (PLEG): container finished" podID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" containerID="6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582" exitCode=0 Jul 09 18:15:40.512014 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.511974 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" Jul 09 18:15:40.512487 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.512007 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" event={"ID":"ad1ba2bb-d444-4e1a-b618-39b92ab7be99","Type":"ContainerDied","Data":"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582"} Jul 09 18:15:40.512487 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.512058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8" event={"ID":"ad1ba2bb-d444-4e1a-b618-39b92ab7be99","Type":"ContainerDied","Data":"72d48591ab7247cb6c6c1223416ec4cf1b65e8134362c56a7889a21fcc07cd64"} Jul 09 18:15:40.512487 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.512079 2577 scope.go:117] "RemoveContainer" containerID="6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582" Jul 09 18:15:40.520339 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.520321 2577 scope.go:117] "RemoveContainer" containerID="6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582" Jul 09 18:15:40.520629 ip-10-0-130-12 kubenswrapper[2577]: E0709 18:15:40.520597 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582\": container with ID starting with 6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582 not found: ID does not exist" containerID="6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582" Jul 09 18:15:40.520731 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.520635 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582"} err="failed to get container status \"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582\": rpc error: code = NotFound desc = could not find container \"6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582\": container with ID starting with 6fbea538a58c2992c6175eb95f9865dcfd3ac55f04bcfd8bc04d566489667582 not found: ID does not exist" Jul 09 18:15:40.530841 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.530816 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:15:40.537304 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:40.537284 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0b9f9-54559cb7c5-87jv8"] Jul 09 18:15:42.074724 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:42.074688 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" path="/var/lib/kubelet/pods/ad1ba2bb-d444-4e1a-b618-39b92ab7be99/volumes" Jul 09 18:15:51.586709 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:51.586674 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-btwq7_c11272d8-2657-41bc-b218-c2565d8bc5a4/manager/0.log" Jul 09 18:15:51.722271 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:51.722236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57f76b769b-wj56k_6015bcce-4bf6-4c3c-82b4-e81c7cec89b3/manager/0.log" Jul 09 18:15:52.088381 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:52.088323 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c8444b686-2m6z7_c8b91e09-c552-43a5-b56f-1048f059c5f0/manager/0.log" Jul 09 18:15:52.094802 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:52.094780 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c8444b686-2m6z7_c8b91e09-c552-43a5-b56f-1048f059c5f0/copy-manifests/0.log" Jul 09 18:15:52.738039 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:52.738005 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-b55d88699-mw5hg_b8e78416-1b08-466a-8c71-fde54390e0de/maas-api/0.log" Jul 09 18:15:53.225624 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:53.225593 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mvb4b_5fa267d3-2459-4dc7-a31d-ded33042ec42/postgres/0.log" Jul 09 18:15:55.080484 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:55.080434 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jlhkv_515e4ca0-4ab3-42b8-ba04-e74f4a367b3e/registry-server/0.log" Jul 09 18:15:55.492301 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:55.492194 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-79h2v_217b8486-1467-4aa4-8b04-db9a3a7274b3/manager/0.log" Jul 09 18:15:56.717998 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:56.717966 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54d66b6fb6-lfbzr_646b2559-7599-4b05-b2ab-fa37a13626fd/router/0.log" Jul 09 18:15:57.084979 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:57.084949 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nrrxh_37f16509-ee31-495a-a334-434be73a1b9e/main/0.log" Jul 09 18:15:57.599732 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:57.599702 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nzckl_a46f8d48-a311-495c-a907-e384258ecfa9/main/0.log" Jul 09 18:15:57.720792 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:15:57.720763 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-bf7hh_2fa76ab5-ebbe-480e-923e-eacc1badd7f6/main/0.log" Jul 09 18:16:04.436733 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:04.436696 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2dbb5_21fe8f0d-338f-4886-bed2-a7ec244552ba/global-pull-secret-syncer/0.log" Jul 09 18:16:04.576979 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:04.576937 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-csfxj_d7c79980-b4ef-4f51-9c50-526b6b290205/konnectivity-agent/0.log" Jul 09 18:16:04.660122 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:04.660093 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-12.ec2.internal_df20e6c48b1fb1cb9b5d2a27957e95a4/haproxy/0.log" Jul 09 18:16:10.404329 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:10.404300 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jlhkv_515e4ca0-4ab3-42b8-ba04-e74f4a367b3e/registry-server/0.log" Jul 09 18:16:10.642934 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:10.642902 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-79h2v_217b8486-1467-4aa4-8b04-db9a3a7274b3/manager/0.log" Jul 09 18:16:12.371211 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:12.371178 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-jfltc_10d58688-da25-40ec-9950-986e594fd257/cluster-monitoring-operator/0.log" Jul 09 18:16:12.529692 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:12.529660 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gvmn6_92b304e3-18a1-4981-9a25-74d3c9df4c2c/node-exporter/0.log" Jul 09 18:16:12.551427 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:12.551394 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gvmn6_92b304e3-18a1-4981-9a25-74d3c9df4c2c/kube-rbac-proxy/0.log" Jul 09 18:16:12.583127 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:12.583100 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gvmn6_92b304e3-18a1-4981-9a25-74d3c9df4c2c/init-textfile/0.log" Jul 09 18:16:14.338838 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.338803 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-jjwkq_4fd31e0c-76d6-4d77-8f6e-a9ef47b73bfa/networking-console-plugin/0.log" Jul 09 18:16:14.658440 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658330 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj"] Jul 09 18:16:14.658842 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658814 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" containerName="maas-api" Jul 09 18:16:14.658842 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" containerName="maas-api" Jul 09 18:16:14.659050 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658867 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef73c41c-cade-4976-a669-11b67b9baf66" containerName="maas-api" Jul 09 18:16:14.659050 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658873 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef73c41c-cade-4976-a669-11b67b9baf66" containerName="maas-api" Jul 09 18:16:14.659050 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658950 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef73c41c-cade-4976-a669-11b67b9baf66" containerName="maas-api" Jul 09 18:16:14.659050 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.658961 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad1ba2bb-d444-4e1a-b618-39b92ab7be99" containerName="maas-api" Jul 09 18:16:14.662217 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.662197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.665050 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.665031 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w85k\"/\"kube-root-ca.crt\"" Jul 09 18:16:14.665887 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.665870 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4w85k\"/\"default-dockercfg-vct85\"" Jul 09 18:16:14.665887 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.665883 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w85k\"/\"openshift-service-ca.crt\"" Jul 09 18:16:14.676926 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.676905 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj"] Jul 09 18:16:14.857434 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.857402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-proc\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.857605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.857456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-lib-modules\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.857605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.857514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-podres\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.857605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.857556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-sys\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.857605 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.857590 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6522g\" (UniqueName: \"kubernetes.io/projected/1673fa56-7049-4ea4-9787-3d4b33b92fec-kube-api-access-6522g\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959091 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-proc\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959091 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-lib-modules\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959091 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-podres\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959091 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-sys\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959342 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959129 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6522g\" (UniqueName: \"kubernetes.io/projected/1673fa56-7049-4ea4-9787-3d4b33b92fec-kube-api-access-6522g\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959342 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959147 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-proc\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959342 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959184 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-podres\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959342 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959189 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-lib-modules\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.959342 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.959223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1673fa56-7049-4ea4-9787-3d4b33b92fec-sys\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.966316 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.966289 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/1.log" Jul 09 18:16:14.967549 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.967528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6522g\" (UniqueName: \"kubernetes.io/projected/1673fa56-7049-4ea4-9787-3d4b33b92fec-kube-api-access-6522g\") pod \"perf-node-gather-daemonset-8kcrj\" (UID: \"1673fa56-7049-4ea4-9787-3d4b33b92fec\") " pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.971958 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.971906 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:14.975261 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:14.975180 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rq8nf_341cad21-7443-4c36-9c1e-29779538a458/console-operator/2.log" Jul 09 18:16:15.110343 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:15.110297 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj"] Jul 09 18:16:15.112613 ip-10-0-130-12 kubenswrapper[2577]: W0709 18:16:15.112580 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1673fa56_7049_4ea4_9787_3d4b33b92fec.slice/crio-e48f65d9ea0c1064d7333503d841f3c91368403d02989d14b8fc40ca2a491aab WatchSource:0}: Error finding container e48f65d9ea0c1064d7333503d841f3c91368403d02989d14b8fc40ca2a491aab: Status 404 returned error can't find the container with id e48f65d9ea0c1064d7333503d841f3c91368403d02989d14b8fc40ca2a491aab Jul 09 18:16:15.681939 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:15.681901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" event={"ID":"1673fa56-7049-4ea4-9787-3d4b33b92fec","Type":"ContainerStarted","Data":"52f96af1419546f12d3df468a95253d855d9e96a5efc73d1406ece5f5ad660e0"} Jul 09 18:16:15.682450 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:15.681963 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" event={"ID":"1673fa56-7049-4ea4-9787-3d4b33b92fec","Type":"ContainerStarted","Data":"e48f65d9ea0c1064d7333503d841f3c91368403d02989d14b8fc40ca2a491aab"} Jul 09 18:16:15.682450 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:15.682015 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:15.699801 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:15.699744 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" podStartSLOduration=1.699724304 podStartE2EDuration="1.699724304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:16:15.697436252 +0000 UTC m=+2918.146488680" watchObservedRunningTime="2026-07-09 18:16:15.699724304 +0000 UTC m=+2918.148776705" Jul 09 18:16:16.073848 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:16.073817 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-x89wt_2f94944d-8636-4b03-8888-7f7c606524dc/volume-data-source-validator/0.log" Jul 09 18:16:17.132066 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:17.132032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-prsrd_03c4b441-78be-49f8-8090-d0fadd5af852/dns/0.log" Jul 09 18:16:17.152853 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:17.152825 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-prsrd_03c4b441-78be-49f8-8090-d0fadd5af852/kube-rbac-proxy/0.log" Jul 09 18:16:17.262518 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:17.262473 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sjr8l_b83651f0-ec52-46a6-9671-24fef94d000b/dns-node-resolver/0.log" Jul 09 18:16:17.874277 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:17.874232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kjvx5_7d19f4e2-714d-458f-8d05-e5b2b92b03ab/node-ca/0.log" Jul 09 18:16:19.140632 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:19.140600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54d66b6fb6-lfbzr_646b2559-7599-4b05-b2ab-fa37a13626fd/router/0.log" Jul 09 18:16:19.855555 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:19.855514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lbnts_41c280e1-4c69-4a64-895b-68aa2656f849/serve-healthcheck-canary/0.log" Jul 09 18:16:20.388650 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:20.388617 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-66x4d_bedd31c3-14f3-42f6-8da8-133b13e061e3/insights-operator/0.log" Jul 09 18:16:20.389791 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:20.389771 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-66x4d_bedd31c3-14f3-42f6-8da8-133b13e061e3/insights-operator/1.log" Jul 09 18:16:20.515932 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:20.515904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjkw7_9f6f190c-140b-4e27-9d61-d765bd2dfd4f/kube-rbac-proxy/0.log" Jul 09 18:16:20.538649 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:20.538617 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjkw7_9f6f190c-140b-4e27-9d61-d765bd2dfd4f/exporter/0.log" Jul 09 18:16:20.561283 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:20.561257 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjkw7_9f6f190c-140b-4e27-9d61-d765bd2dfd4f/extractor/0.log" Jul 09 18:16:21.696729 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:21.696704 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4w85k/perf-node-gather-daemonset-8kcrj" Jul 09 18:16:22.946454 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:22.946411 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-btwq7_c11272d8-2657-41bc-b218-c2565d8bc5a4/manager/0.log" Jul 09 18:16:23.123332 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:23.123296 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57f76b769b-wj56k_6015bcce-4bf6-4c3c-82b4-e81c7cec89b3/manager/0.log" Jul 09 18:16:23.265820 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:23.265725 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c8444b686-2m6z7_c8b91e09-c552-43a5-b56f-1048f059c5f0/manager/0.log" Jul 09 18:16:23.287533 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:23.287506 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c8444b686-2m6z7_c8b91e09-c552-43a5-b56f-1048f059c5f0/copy-manifests/0.log" Jul 09 18:16:24.839233 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:24.839201 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54d66b6fb6-lfbzr_646b2559-7599-4b05-b2ab-fa37a13626fd/router/0.log" Jul 09 18:16:24.891767 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:24.891683 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6784c458c5-sjtw8_3192ba7a-3fce-4396-8bc2-e1ad70dd960d/manager/0.log" Jul 09 18:16:26.329252 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:26.329219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54d66b6fb6-lfbzr_646b2559-7599-4b05-b2ab-fa37a13626fd/router/0.log" Jul 09 18:16:29.801097 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:29.801068 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xn2m4_7dc70c29-19e7-40f1-a41a-9bcc21f33bf2/migrator/0.log" Jul 09 18:16:29.821742 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:29.821708 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-xn2m4_7dc70c29-19e7-40f1-a41a-9bcc21f33bf2/graceful-termination/0.log" Jul 09 18:16:30.170517 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:30.170484 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-mwdxg_d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b/kube-storage-version-migrator-operator/1.log" Jul 09 18:16:30.172161 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:30.172132 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-mwdxg_d876a62a-e6e9-4d40-8ae7-8ed4bcf7910b/kube-storage-version-migrator-operator/0.log" Jul 09 18:16:31.127634 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.127603 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/kube-multus-additional-cni-plugins/0.log" Jul 09 18:16:31.149276 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.149250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/egress-router-binary-copy/0.log" Jul 09 18:16:31.169852 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.169828 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/cni-plugins/0.log" Jul 09 18:16:31.190189 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.190160 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/bond-cni-plugin/0.log" Jul 09 18:16:31.211668 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.211642 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/routeoverride-cni/0.log" Jul 09 18:16:31.232177 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.232146 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/whereabouts-cni-bincopy/0.log" Jul 09 18:16:31.251973 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.251947 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bfldg_3ad28ce5-02b2-49ad-ab0e-c6d1a914f299/whereabouts-cni/0.log" Jul 09 18:16:31.662850 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.662813 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n9s5d_047648cf-d30a-4556-aa48-5ba980836a26/kube-multus/0.log" Jul 09 18:16:31.736297 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.736265 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t5g2n_431e5c77-b3ad-49b7-87d9-95ee5648429b/network-metrics-daemon/0.log" Jul 09 18:16:31.755057 ip-10-0-130-12 kubenswrapper[2577]: I0709 18:16:31.755031 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t5g2n_431e5c77-b3ad-49b7-87d9-95ee5648429b/kube-rbac-proxy/0.log"