Jul 16 20:31:00.867855 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 20:31:00.867867 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 20:31:00.867901 ip-10-0-135-240 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 20:31:00.868267 ip-10-0-135-240 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 20:31:11.112087 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 20:31:11.112102 ip-10-0-135-240 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 51441dc5726945bfb58ebe8600aa98b2 -- Jul 16 20:33:12.021836 ip-10-0-135-240 systemd[1]: Starting Kubernetes Kubelet... Jul 16 20:33:12.489391 ip-10-0-135-240 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 20:33:12.489391 ip-10-0-135-240 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 20:33:12.489391 ip-10-0-135-240 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 20:33:12.489391 ip-10-0-135-240 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 20:33:12.489391 ip-10-0-135-240 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 20:33:12.491418 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.491298 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 20:33:12.496745 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496731 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.496745 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496744 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496748 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496753 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496757 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496761 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496764 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496767 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496770 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496773 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496777 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496780 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496784 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496786 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496798 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496801 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496804 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496807 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496809 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.496807 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496812 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496815 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496818 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496820 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496823 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496826 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496829 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496832 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496835 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496838 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496840 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496843 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496845 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496848 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496850 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496853 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496856 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496858 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496861 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496864 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.497229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496867 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496869 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496872 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496874 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496876 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496879 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496901 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496906 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496909 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496912 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496914 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496917 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496919 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496921 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496925 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496928 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496931 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496934 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496936 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496939 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.497710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496942 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496944 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496947 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496949 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496952 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496954 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496957 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496959 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496962 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496964 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496967 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496969 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496972 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496974 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496977 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496980 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496982 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496984 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496987 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496990 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496993 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.498189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496995 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.496998 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497000 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497002 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497005 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497008 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497011 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497013 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497016 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497018 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497021 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497023 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497026 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497028 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497031 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497033 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497036 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497038 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497041 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497723 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.498739 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497730 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497735 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497739 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497741 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497745 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497747 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497750 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497752 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497755 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497757 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497761 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497764 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497766 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497769 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497771 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497774 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497777 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497779 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497782 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.499199 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497785 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497788 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497790 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497794 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497798 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497800 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497803 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497806 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497809 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497811 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497814 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497816 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497819 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497821 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497830 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497833 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497836 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497839 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497842 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497845 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.499670 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497847 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497850 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497852 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497855 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497858 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497860 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497864 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497868 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497871 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497874 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497877 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497880 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497883 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497886 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497889 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497892 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497895 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497897 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497900 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497903 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.500130 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497905 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497908 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497910 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497913 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497915 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497918 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497920 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497923 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497925 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497928 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497930 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497933 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497936 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497938 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497941 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497944 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497946 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497949 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497951 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497954 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.500609 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497957 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497960 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497963 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497965 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497968 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497970 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497973 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497975 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497978 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497980 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497983 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497985 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497988 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497990 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497993 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497995 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.497998 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498001 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498003 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498374 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.501081 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498379 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498383 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498386 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498390 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498393 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498395 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498398 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498401 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498404 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498407 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498410 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498412 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498415 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498417 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498420 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498423 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498426 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498428 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498431 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498433 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.501563 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498436 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498438 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498441 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498443 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498446 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498448 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498451 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498453 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498456 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498458 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498460 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498463 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498466 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498468 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498471 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498473 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498476 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498479 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498482 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498484 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.502073 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498486 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498489 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498491 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498494 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498497 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498499 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498502 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498504 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498507 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498511 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498515 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498518 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498521 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498524 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498526 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498528 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498531 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498534 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498537 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.502546 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498541 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498544 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498547 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498549 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498552 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498555 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498558 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498560 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498563 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498566 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498582 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498587 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498590 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498593 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498597 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498600 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498603 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498605 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498608 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.503017 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498610 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498613 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498615 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498618 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498620 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498623 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498625 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498627 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498630 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498633 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498635 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498638 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498641 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498643 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498645 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498648 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498651 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498653 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498655 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.498659 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.503527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498720 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498733 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498741 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498745 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498754 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498758 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498763 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498767 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498770 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498773 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498776 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498780 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498783 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498786 2582 flags.go:64] FLAG: --cgroup-root="" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498789 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498792 2582 flags.go:64] FLAG: --client-ca-file="" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498794 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498797 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498801 2582 flags.go:64] FLAG: --cluster-domain="" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498804 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498807 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498810 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498813 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498817 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 20:33:12.504024 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498820 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498823 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498826 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498829 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498832 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498834 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498837 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498840 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498846 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498849 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498852 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498855 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498859 2582 flags.go:64] FLAG: --enable-server="true" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498863 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498868 2582 flags.go:64] FLAG: --event-burst="100" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498871 2582 flags.go:64] FLAG: --event-qps="50" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498874 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498877 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498880 2582 flags.go:64] FLAG: --eviction-hard="" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498883 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498886 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498889 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498892 2582 flags.go:64] FLAG: --eviction-soft="" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498895 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498897 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 20:33:12.504586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498900 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498903 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498906 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498909 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498912 2582 flags.go:64] FLAG: --feature-gates="" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498915 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498918 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498921 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498924 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498927 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498930 2582 flags.go:64] FLAG: --help="false" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498932 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498935 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498938 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498941 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498947 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498950 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498953 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498956 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498960 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498963 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498966 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498969 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498972 2582 flags.go:64] FLAG: --kube-reserved="" Jul 16 20:33:12.505196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498975 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498977 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498980 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498983 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498986 2582 flags.go:64] FLAG: --lock-file="" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498988 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498991 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498994 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.498999 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499002 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499004 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499007 2582 flags.go:64] FLAG: --logging-format="text" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499010 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499013 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499016 2582 flags.go:64] FLAG: --manifest-url="" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499019 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499023 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499026 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499030 2582 flags.go:64] FLAG: --max-pods="110" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499033 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499036 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499038 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499042 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499046 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499049 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 20:33:12.505775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499052 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499058 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499063 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499066 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499069 2582 flags.go:64] FLAG: --pod-cidr="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499073 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499078 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499081 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499084 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499086 2582 flags.go:64] FLAG: --port="10250" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499089 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499092 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-021d1d6bc53e739c0" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499095 2582 flags.go:64] FLAG: --qos-reserved="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499098 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499101 2582 flags.go:64] FLAG: --register-node="true" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499104 2582 flags.go:64] FLAG: --register-with-taints="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499107 2582 flags.go:64] FLAG: --registry-burst="10" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499110 2582 flags.go:64] FLAG: --registry-qps="5" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499113 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499115 2582 flags.go:64] FLAG: --reserved-memory="" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499119 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499122 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499125 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499128 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499130 2582 flags.go:64] FLAG: --runonce="false" Jul 16 20:33:12.506439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499133 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499136 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499139 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499142 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499144 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499147 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499151 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499154 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499157 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499161 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499164 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499167 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499170 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499173 2582 flags.go:64] FLAG: --system-cgroups="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499176 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499181 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499184 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499187 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499191 2582 flags.go:64] FLAG: --tls-min-version="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499193 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499196 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499199 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499202 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499205 2582 flags.go:64] FLAG: --v="2" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499209 2582 flags.go:64] FLAG: --version="false" Jul 16 20:33:12.507102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499212 2582 flags.go:64] FLAG: --vmodule="" Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499216 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.499219 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499324 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499327 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499331 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499334 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499337 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499340 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499342 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499345 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499348 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499350 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499354 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499357 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499359 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499363 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499366 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499368 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499371 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.507762 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499373 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499377 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499379 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499382 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499384 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499387 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499389 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499392 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499394 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499397 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499399 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499402 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499405 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499407 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499410 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499412 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499415 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499419 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499422 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.508239 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499426 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499429 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499431 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499434 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499438 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499441 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499445 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499448 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499452 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499456 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499459 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499462 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499465 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499467 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499470 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499472 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499475 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499478 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499480 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.508710 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499482 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499485 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499487 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499490 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499492 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499495 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499497 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499500 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499502 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499505 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499507 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499510 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499512 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499514 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499517 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499519 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499522 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499525 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499527 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499531 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.509177 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499534 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499536 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499541 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499543 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499546 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499549 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499551 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499554 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499556 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499559 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499562 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499564 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499567 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499585 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499588 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499590 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499593 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499596 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499598 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499601 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.509685 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499603 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499606 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499608 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.499611 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.500794 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.507932 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.507949 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.507969 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.507978 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508040 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508045 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508049 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508052 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508055 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508057 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.510269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508060 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508063 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508066 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508068 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508071 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508073 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508076 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508078 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508080 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508083 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508085 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508088 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508090 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508093 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508096 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508098 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508101 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508103 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508106 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508109 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.510872 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508112 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508114 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508117 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508126 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508130 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508134 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508137 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508140 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508143 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508146 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508148 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508151 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508153 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508156 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508158 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508161 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508163 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508166 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508168 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.511465 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508171 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508173 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508176 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508178 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508181 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508183 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508186 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508189 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508192 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508194 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508197 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508200 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508202 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508205 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508207 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508210 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508212 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508222 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508225 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508227 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.511944 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508229 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508232 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508234 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508237 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508239 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508256 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508260 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508264 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508267 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508270 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508273 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508276 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508279 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508281 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508284 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508287 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508289 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508292 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508295 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508303 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.512425 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508306 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508308 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508311 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508313 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508316 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508318 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508321 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508323 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508326 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508328 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508338 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508342 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508344 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508348 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.512958 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.508353 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508498 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508506 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508509 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508512 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508515 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508518 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508521 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508523 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508526 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508528 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508531 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508533 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508536 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508538 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508541 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508543 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508546 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508548 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 20:33:12.513303 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508551 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508554 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508556 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508559 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508561 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508564 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508566 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508582 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508585 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508587 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508595 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508598 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508601 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508603 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508605 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508608 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508610 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508613 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508617 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508620 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 20:33:12.513778 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508623 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508625 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508628 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508631 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508633 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508636 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508638 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508641 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508643 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508646 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508649 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508651 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508654 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508656 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508659 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508661 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508664 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508666 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508669 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508671 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508674 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 20:33:12.514269 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508676 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508679 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508687 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508689 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508692 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508694 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508697 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508699 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508702 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508704 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508707 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508709 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508712 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508715 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508719 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508722 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508725 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508728 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508730 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 20:33:12.514820 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508733 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508736 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508739 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508743 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508746 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508749 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508751 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508754 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508756 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508759 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508762 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508764 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508767 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508769 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508771 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508774 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508782 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508784 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508787 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 20:33:12.515273 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508789 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 20:33:12.515738 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:12.508792 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 20:33:12.515738 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.508796 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 20:33:12.515738 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.508914 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 20:33:12.515738 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.514125 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 20:33:12.516129 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.516113 2582 server.go:1019] "Starting client certificate rotation" Jul 16 20:33:12.516233 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.516218 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 20:33:12.516265 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.516255 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 20:33:12.518606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.518588 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 20:33:12.520053 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.520039 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 20:33:12.529176 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.529159 2582 log.go:25] "Validated CRI v1 runtime API" Jul 16 20:33:12.531500 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.531488 2582 log.go:25] "Validated CRI v1 image API" Jul 16 20:33:12.532921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.532905 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 20:33:12.538320 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.538303 2582 fs.go:135] Filesystem UUIDs: map[4eba1906-4477-4b91-b20a-e849207c90e9:/dev/nvme0n1p4 728ef4b0-1fe9-491b-b35d-33325fc2b42a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 16 20:33:12.538384 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.538320 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 20:33:12.544794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.544688 2582 manager.go:216] Machine: {Timestamp:2026-07-16 20:33:12.543166206 +0000 UTC m=+0.385017344 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3053719 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec267dce0a128f787741ff17278e0db0 SystemUUID:ec267dce-0a12-8f78-7741-ff17278e0db0 BootID:51441dc5-7269-45bf-b58e-be8600aa98b2 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:87:51:b5:de:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:87:51:b5:de:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:35:a8:f9:bd:c2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 20:33:12.544794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.544786 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 20:33:12.544953 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.544853 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 20:33:12.547595 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.547564 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 20:33:12.548294 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.548274 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 20:33:12.548434 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.548297 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 20:33:12.549747 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.549736 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 20:33:12.549747 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.549749 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 20:33:12.549821 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.549756 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 20:33:12.549821 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.549766 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 20:33:12.551377 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.551367 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 20:33:12.554939 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.554928 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 16 20:33:12.555038 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555028 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 20:33:12.555080 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555057 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 16 20:33:12.555080 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555074 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 20:33:12.555160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555091 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 20:33:12.555160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555100 2582 kubelet.go:398] "Adding apiserver pod source" Jul 16 20:33:12.555160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555110 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 20:33:12.555417 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555407 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 20:33:12.555458 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.555424 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 20:33:12.559101 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.559085 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 20:33:12.560707 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.560693 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 20:33:12.560764 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.560723 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 20:33:12.561489 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561478 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 20:33:12.561522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561493 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 20:33:12.561522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561499 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 20:33:12.561522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561505 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 20:33:12.561522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561513 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 20:33:12.561522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561518 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561526 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561531 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561537 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561542 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561551 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 20:33:12.561668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.561556 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 20:33:12.562638 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.562628 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 20:33:12.562638 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.562638 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 20:33:12.568021 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.567996 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 20:33:12.568694 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.568671 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 20:33:12.568759 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.568666 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 20:33:12.569360 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.569346 2582 server.go:1268] "Started kubelet" Jul 16 20:33:12.569461 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.569424 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 20:33:12.569521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.569425 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 20:33:12.569521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.569501 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 20:33:12.570154 ip-10-0-135-240 systemd[1]: Started Kubernetes Kubelet. Jul 16 20:33:12.570563 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.570545 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 20:33:12.571875 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.571860 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 16 20:33:12.574701 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.574684 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 20:33:12.576119 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.575089 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18c2df5ceb457548 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-07-16 20:33:12.569320776 +0000 UTC m=+0.411171894,LastTimestamp:2026-07-16 20:33:12.569320776 +0000 UTC m=+0.411171894,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}" Jul 16 20:33:12.578790 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.578772 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 20:33:12.578790 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.578778 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 20:33:12.579144 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.578879 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.579410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579394 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 20:33:12.579410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579410 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 20:33:12.579537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579417 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 20:33:12.579537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579524 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 16 20:33:12.579537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579535 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 16 20:33:12.579687 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579593 2582 factory.go:55] Registering systemd factory Jul 16 20:33:12.579687 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579609 2582 factory.go:223] Registration of the systemd container factory successfully Jul 16 20:33:12.579838 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579823 2582 factory.go:153] Registering CRI-O factory Jul 16 20:33:12.579914 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579840 2582 factory.go:223] Registration of the crio container factory successfully Jul 16 20:33:12.579914 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579888 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 20:33:12.579914 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579912 2582 factory.go:103] Registering Raw factory Jul 16 20:33:12.580048 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.579923 2582 manager.go:1196] Started watching for new ooms in manager Jul 16 20:33:12.580234 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.580224 2582 manager.go:349] Starting recovery of all containers Jul 16 20:33:12.585283 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.585255 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 20:33:12.585366 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.585323 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 20:33:12.592119 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.592098 2582 manager.go:354] Recovery completed Jul 16 20:33:12.594217 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.594196 2582 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 16 20:33:12.597207 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.597195 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.599136 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599120 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.599204 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599147 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.599204 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599157 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.599405 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599390 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7q67g" Jul 16 20:33:12.599677 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599664 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 20:33:12.599677 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599671 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.599677 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599676 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 20:33:12.599780 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.599690 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 16 20:33:12.600272 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600260 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.600315 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600284 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.600315 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600295 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.600967 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600955 2582 policy_none.go:49] "None policy: Start" Jul 16 20:33:12.601002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600970 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 20:33:12.601002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.600980 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 20:33:12.601209 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.601127 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18c2df5ced0c658e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-07-16 20:33:12.59913563 +0000 UTC m=+0.440986751,LastTimestamp:2026-07-16 20:33:12.59913563 +0000 UTC m=+0.440986751,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}" Jul 16 20:33:12.602190 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.602179 2582 policy_none.go:47] "Start" Jul 16 20:33:12.605135 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.605120 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7q67g" Jul 16 20:33:12.610870 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.610811 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18c2df5ced0ca5ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-240.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-07-16 20:33:12.599152046 +0000 UTC m=+0.441003164,LastTimestamp:2026-07-16 20:33:12.599152046 +0000 UTC m=+0.441003164,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}" Jul 16 20:33:12.639167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639075 2582 manager.go:337] "Starting Device Plugin manager" Jul 16 20:33:12.639247 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.639179 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 20:33:12.639247 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639188 2582 server.go:87] "Starting device plugin registration server" Jul 16 20:33:12.639427 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639415 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 20:33:12.639485 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639426 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 20:33:12.639532 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639521 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 20:33:12.639608 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639599 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 20:33:12.639643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.639608 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 20:33:12.640006 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.639976 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 20:33:12.640069 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.640040 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.715099 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.715070 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 20:33:12.716296 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.716163 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 20:33:12.716296 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.716193 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 20:33:12.716296 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.716215 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 20:33:12.716296 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.716244 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 20:33:12.718528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.718510 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:12.740191 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.740145 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.740977 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.740962 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.741043 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.740995 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.741043 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.741005 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.741043 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.741026 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.749651 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.749638 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.749700 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.749657 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-240.ec2.internal\": node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.772971 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.772954 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.816974 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.816956 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"] Jul 16 20:33:12.817041 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.817006 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.818384 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.818371 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.818445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.818391 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.818445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.818403 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.819578 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.819554 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.819676 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.819663 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.819711 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.819691 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.820360 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820344 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.820440 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820366 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.820440 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820378 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.820680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820627 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.820680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820649 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.820680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.820660 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.822087 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.822071 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.822155 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.822100 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:12.822732 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.822714 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:12.822810 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.822740 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:12.822810 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.822752 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:12.848065 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.848047 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.852112 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.852100 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.873362 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.873349 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.880769 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.880753 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.880836 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.880771 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.880836 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.880786 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f3a70c0e179a36534f61d6399c4980f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"9f3a70c0e179a36534f61d6399c4980f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.974170 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:12.974146 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:12.981446 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.981512 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.981512 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f3a70c0e179a36534f61d6399c4980f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"9f3a70c0e179a36534f61d6399c4980f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.981512 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981494 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f3a70c0e179a36534f61d6399c4980f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"9f3a70c0e179a36534f61d6399c4980f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.981619 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981511 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:12.981619 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:12.981518 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bfe31dea644c334a88e35254b8a1e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"64bfe31dea644c334a88e35254b8a1e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:13.074725 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.074671 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.150042 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.150015 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:13.155989 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.155975 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:13.174957 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.174935 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.275399 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.275378 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.375791 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.375746 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.453951 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.453932 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:13.476392 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.476369 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.514734 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.514716 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 20:33:13.515056 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.514828 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 20:33:13.515056 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.514844 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 20:33:13.577052 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.577033 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.586935 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:13.586906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3a70c0e179a36534f61d6399c4980f.slice/crio-85d9ea03a912665521dd44eeecfff71086bca6a9ba4e21e739ae882bf751c9a9 WatchSource:0}: Error finding container 85d9ea03a912665521dd44eeecfff71086bca6a9ba4e21e739ae882bf751c9a9: Status 404 returned error can't find the container with id 85d9ea03a912665521dd44eeecfff71086bca6a9ba4e21e739ae882bf751c9a9 Jul 16 20:33:13.591272 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:13.591249 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bfe31dea644c334a88e35254b8a1e2.slice/crio-693211b859a0aa0d425e51de6b449c51ef5040e5cdd2d47d14b50ed3f9d76a6d WatchSource:0}: Error finding container 693211b859a0aa0d425e51de6b449c51ef5040e5cdd2d47d14b50ed3f9d76a6d: Status 404 returned error can't find the container with id 693211b859a0aa0d425e51de6b449c51ef5040e5cdd2d47d14b50ed3f9d76a6d Jul 16 20:33:13.591377 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.591363 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 20:33:13.610002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.609968 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 20:28:12 +0000 UTC" deadline="2028-01-31 15:26:26.609031521 +0000 UTC" Jul 16 20:33:13.610002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.609998 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13530h53m12.999035891s" Jul 16 20:33:13.677203 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.677167 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.719463 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.719424 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"9f3a70c0e179a36534f61d6399c4980f","Type":"ContainerStarted","Data":"85d9ea03a912665521dd44eeecfff71086bca6a9ba4e21e739ae882bf751c9a9"} Jul 16 20:33:13.720487 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.720466 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"64bfe31dea644c334a88e35254b8a1e2","Type":"ContainerStarted","Data":"693211b859a0aa0d425e51de6b449c51ef5040e5cdd2d47d14b50ed3f9d76a6d"} Jul 16 20:33:13.777618 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:13.777600 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 16 20:33:13.790922 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.790907 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 20:33:13.791806 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.791792 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 16 20:33:13.791879 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.791818 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 20:33:13.791879 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.791832 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 16 20:33:13.852810 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.852791 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:13.879444 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.879428 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 16 20:33:13.890966 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.890952 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 20:33:13.892919 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.892908 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 16 20:33:13.900688 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:13.900676 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 20:33:14.099519 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.099446 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:14.385454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.385390 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:14.556469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.556442 2582 apiserver.go:52] "Watching apiserver" Jul 16 20:33:14.562973 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.562950 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 20:33:14.563369 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.563349 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-szbgs","openshift-network-operator/iptables-alerter-grqmq","kube-system/konnectivity-agent-wgj47","openshift-image-registry/node-ca-npv56","openshift-multus/multus-5nssp","openshift-multus/network-metrics-daemon-9sqrn","openshift-network-diagnostics/network-check-target-pjcxx","openshift-ovn-kubernetes/ovnkube-node-r5n26","kube-system/global-pull-secret-syncer-84n8n","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67","openshift-cluster-node-tuning-operator/tuned-zjfn7","openshift-dns/node-resolver-gvsbw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"] Jul 16 20:33:14.564814 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.564797 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:14.564906 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.564869 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:14.566742 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.566724 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.567895 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.567872 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.569079 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.568872 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.569079 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.568921 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nffj2\"" Jul 16 20:33:14.569079 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.568972 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.569268 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.569155 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.569947 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.569847 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 20:33:14.569947 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.569904 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8pqsb\"" Jul 16 20:33:14.569947 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.569911 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 20:33:14.570628 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.570613 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.571068 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.571051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.571445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.571431 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fzshc\"" Jul 16 20:33:14.571858 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.571805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.571938 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.571876 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.572947 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573056 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573087 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jrftb\"" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573087 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573155 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.573167 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573163 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.573453 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573264 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 20:33:14.573453 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.573356 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.573453 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.573411 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:14.575644 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.575584 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.577037 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.577017 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.577608 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.577591 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 20:33:14.577714 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.577693 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-865jb\"" Jul 16 20:33:14.577791 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.577694 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 20:33:14.578538 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.578522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.579123 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579099 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 20:33:14.579123 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579107 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 20:33:14.579282 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579125 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.579282 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579146 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.579282 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579107 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l55tb\"" Jul 16 20:33:14.579937 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.579922 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.580537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580519 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jppmm\"" Jul 16 20:33:14.580641 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580602 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 20:33:14.580641 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580609 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 20:33:14.580981 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580872 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.580981 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580886 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.580981 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580896 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 20:33:14.580981 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.580970 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 20:33:14.581493 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.581340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.581884 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.581865 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.581968 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.581887 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 20:33:14.582023 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.581984 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.582023 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.582003 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7shrg\"" Jul 16 20:33:14.583987 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.583968 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:14.584077 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.584014 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:14.584077 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.584030 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r7drx\"" Jul 16 20:33:14.588266 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkr2v\" (UniqueName: \"kubernetes.io/projected/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-kube-api-access-mkr2v\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.588371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588274 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-os-release\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.588371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-kubelet\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588320 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-etc-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588341 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-socket-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.588371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-etc-kubernetes\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588404 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-system-cni-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588425 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-os-release\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588445 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-env-overrides\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-script-lib\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptcv\" (UniqueName: \"kubernetes.io/projected/7c6e688a-4008-41e9-970b-115e17aff958-kube-api-access-sptcv\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.588643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588643 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9b15667-eba3-4608-a2f7-2c955b9705dc-tmp-dir\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-multus\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a4ed8fd-86de-40e9-b633-280c9b429ec2-agent-certs\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a62974f8-89b7-470c-a4e7-942a536e9627-iptables-alerter-script\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-sys-fs\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-cnibin\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwbv8\" (UniqueName: \"kubernetes.io/projected/62d02010-e6f6-4108-9904-2e1e32aa50de-kube-api-access-cwbv8\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-bin\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-socket-dir-parent\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a62974f8-89b7-470c-a4e7-942a536e9627-host-slash\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-var-lib-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.588926 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588940 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-registration-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf7lb\" (UniqueName: \"kubernetes.io/projected/b9b15667-eba3-4608-a2f7-2c955b9705dc-kube-api-access-zf7lb\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.588982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-kubelet-config\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-slash\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589017 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-node-log\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-netns\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-dbus\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-serviceca\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmjh4\" (UniqueName: \"kubernetes.io/projected/fe300f32-518a-4ff9-9700-d7ffd99864bd-kube-api-access-bmjh4\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589114 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ed8fd-86de-40e9-b633-280c9b429ec2-konnectivity-ca\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589158 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589177 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-kubelet\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-binary-copy\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.589324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589303 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-system-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589331 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-cni-binary-copy\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589357 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-multus-certs\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kttds\" (UniqueName: \"kubernetes.io/projected/a62974f8-89b7-470c-a4e7-942a536e9627-kube-api-access-kttds\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-systemd-units\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589523 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-systemd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-host\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-daemon-config\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-netd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589656 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-ovn\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9b15667-eba3-4608-a2f7-2c955b9705dc-hosts-file\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8tsz\" (UniqueName: \"kubernetes.io/projected/c9238df2-51ed-4a8b-870b-15cf82c175df-kube-api-access-b8tsz\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.589954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589791 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-etc-selinux\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-k8s-cni-cncf-io\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-hostroot\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-config\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-device-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-cnibin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-bin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-conf-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-netns\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.589990 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-log-socket\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.590012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8889d89f-ec52-4033-b2ca-58192f929616-ovn-node-metrics-cert\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.590032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wm82\" (UniqueName: \"kubernetes.io/projected/8889d89f-ec52-4033-b2ca-58192f929616-kube-api-access-2wm82\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.590052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.590601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.590204 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 20:33:14.613179 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.613159 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5s28x" Jul 16 20:33:14.618942 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.618924 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5s28x" Jul 16 20:33:14.680640 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.680591 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 20:33:14.690997 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.690977 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-k8s-cni-cncf-io\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.690997 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691003 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-hostroot\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691018 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-config\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-device-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691080 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-k8s-cni-cncf-io\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691082 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-cnibin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-hostroot\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691125 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-cnibin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691134 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-device-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691139 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-bin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-conf-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-netns\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691180 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-bin\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-log-socket\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-conf-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-netns\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691235 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8889d89f-ec52-4033-b2ca-58192f929616-ovn-node-metrics-cert\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691258 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-log-socket\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wm82\" (UniqueName: \"kubernetes.io/projected/8889d89f-ec52-4033-b2ca-58192f929616-kube-api-access-2wm82\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691310 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkr2v\" (UniqueName: \"kubernetes.io/projected/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-kube-api-access-mkr2v\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-os-release\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691337 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-kubelet\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-etc-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.691449 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691397 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-socket-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-kubelet\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691423 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-var-lib-kubelet\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691444 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-etc-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691449 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-tuned\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-os-release\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-etc-kubernetes\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691514 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-etc-kubernetes\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-system-cni-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-socket-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691552 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-os-release\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.691590 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-config\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-env-overrides\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691635 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-os-release\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.692325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691588 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-system-cni-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.691678 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:15.191627201 +0000 UTC m=+3.033478307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691671 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-script-lib\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691721 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptcv\" (UniqueName: \"kubernetes.io/projected/7c6e688a-4008-41e9-970b-115e17aff958-kube-api-access-sptcv\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9b15667-eba3-4608-a2f7-2c955b9705dc-tmp-dir\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-multus\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a4ed8fd-86de-40e9-b633-280c9b429ec2-agent-certs\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a62974f8-89b7-470c-a4e7-942a536e9627-iptables-alerter-script\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691824 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-sys-fs\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691845 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-cni-multus\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsfvv\" (UniqueName: \"kubernetes.io/projected/2ae65159-0ebd-483a-8612-b9bf5b7b227f-kube-api-access-dsfvv\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691975 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-env-overrides\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691984 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-sys-fs\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.691978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-cnibin\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692013 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-cnibin\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.693084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwbv8\" (UniqueName: \"kubernetes.io/projected/62d02010-e6f6-4108-9904-2e1e32aa50de-kube-api-access-cwbv8\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-bin\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692168 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-run\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8889d89f-ec52-4033-b2ca-58192f929616-ovnkube-script-lib\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-socket-dir-parent\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692220 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-bin\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a62974f8-89b7-470c-a4e7-942a536e9627-host-slash\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-var-lib-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-registration-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf7lb\" (UniqueName: \"kubernetes.io/projected/b9b15667-eba3-4608-a2f7-2c955b9705dc-kube-api-access-zf7lb\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-kubelet-config\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-slash\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692394 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-socket-dir-parent\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9b15667-eba3-4608-a2f7-2c955b9705dc-tmp-dir\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-node-log\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.693845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-netns\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692482 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692487 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a62974f8-89b7-470c-a4e7-942a536e9627-iptables-alerter-script\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692500 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-kubelet-config\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a62974f8-89b7-470c-a4e7-942a536e9627-host-slash\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-dbus\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-registration-dir\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-serviceca\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692612 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-slash\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692614 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmjh4\" (UniqueName: \"kubernetes.io/projected/fe300f32-518a-4ff9-9700-d7ffd99864bd-kube-api-access-bmjh4\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692615 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5556d69e-b310-4298-888e-5f47489a878b-dbus\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692656 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-var-lib-openvswitch\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692683 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-netns\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692687 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-node-log\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692834 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ed8fd-86de-40e9-b633-280c9b429ec2-konnectivity-ca\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.694445 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-modprobe-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692974 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-serviceca\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-kubelet\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.692994 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-run-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-binary-copy\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693031 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-var-lib-kubelet\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-lib-modules\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693148 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-conf\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-host\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-system-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-cni-binary-copy\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693264 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-multus-certs\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-system-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693321 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ed8fd-86de-40e9-b633-280c9b429ec2-konnectivity-ca\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.695114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62d02010-e6f6-4108-9904-2e1e32aa50de-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693358 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kttds\" (UniqueName: \"kubernetes.io/projected/a62974f8-89b7-470c-a4e7-942a536e9627-kube-api-access-kttds\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-host-run-multus-certs\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-systemd-units\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-systemd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-host\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-daemon-config\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-netd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693505 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-systemd\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-cni-binary-copy\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-sys\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-tmp\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693659 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-cni-netd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-host\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693721 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-cni-dir\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-ovn\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.695662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-systemd-units\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693786 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-kubernetes\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693806 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-systemd\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9b15667-eba3-4608-a2f7-2c955b9705dc-hosts-file\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-cni-binary-copy\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9b15667-eba3-4608-a2f7-2c955b9705dc-hosts-file\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693853 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62d02010-e6f6-4108-9904-2e1e32aa50de-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693859 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8tsz\" (UniqueName: \"kubernetes.io/projected/c9238df2-51ed-4a8b-870b-15cf82c175df-kube-api-access-b8tsz\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.693871 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-run-ovn\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.693926 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.693982 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:15.19396357 +0000 UTC m=+3.035814673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694012 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe300f32-518a-4ff9-9700-d7ffd99864bd-multus-daemon-config\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-etc-selinux\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694024 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8889d89f-ec52-4033-b2ca-58192f929616-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysconfig\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c6e688a-4008-41e9-970b-115e17aff958-etc-selinux\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.696202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.694939 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8889d89f-ec52-4033-b2ca-58192f929616-ovn-node-metrics-cert\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.696774 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.695053 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a4ed8fd-86de-40e9-b633-280c9b429ec2-agent-certs\") pod \"konnectivity-agent-wgj47\" (UID: \"3a4ed8fd-86de-40e9-b633-280c9b429ec2\") " pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.699000 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.698975 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkr2v\" (UniqueName: \"kubernetes.io/projected/5d55b6a6-cbd1-41e1-97fc-80c64efeca7b-kube-api-access-mkr2v\") pod \"node-ca-npv56\" (UID: \"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b\") " pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.701921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.701902 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wm82\" (UniqueName: \"kubernetes.io/projected/8889d89f-ec52-4033-b2ca-58192f929616-kube-api-access-2wm82\") pod \"ovnkube-node-r5n26\" (UID: \"8889d89f-ec52-4033-b2ca-58192f929616\") " pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.703391 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.703373 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:14.703476 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.703396 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:14.703476 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.703408 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:14.703476 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:14.703459 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:15.203445726 +0000 UTC m=+3.045296843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:14.704908 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.704889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwbv8\" (UniqueName: \"kubernetes.io/projected/62d02010-e6f6-4108-9904-2e1e32aa50de-kube-api-access-cwbv8\") pod \"multus-additional-cni-plugins-szbgs\" (UID: \"62d02010-e6f6-4108-9904-2e1e32aa50de\") " pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.705657 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.705617 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptcv\" (UniqueName: \"kubernetes.io/projected/7c6e688a-4008-41e9-970b-115e17aff958-kube-api-access-sptcv\") pod \"aws-ebs-csi-driver-node-44g67\" (UID: \"7c6e688a-4008-41e9-970b-115e17aff958\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.706316 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.706297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmjh4\" (UniqueName: \"kubernetes.io/projected/fe300f32-518a-4ff9-9700-d7ffd99864bd-kube-api-access-bmjh4\") pod \"multus-5nssp\" (UID: \"fe300f32-518a-4ff9-9700-d7ffd99864bd\") " pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.706371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.706316 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf7lb\" (UniqueName: \"kubernetes.io/projected/b9b15667-eba3-4608-a2f7-2c955b9705dc-kube-api-access-zf7lb\") pod \"node-resolver-gvsbw\" (UID: \"b9b15667-eba3-4608-a2f7-2c955b9705dc\") " pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.706824 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.706807 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8tsz\" (UniqueName: \"kubernetes.io/projected/c9238df2-51ed-4a8b-870b-15cf82c175df-kube-api-access-b8tsz\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:14.707251 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.707232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kttds\" (UniqueName: \"kubernetes.io/projected/a62974f8-89b7-470c-a4e7-942a536e9627-kube-api-access-kttds\") pod \"iptables-alerter-grqmq\" (UID: \"a62974f8-89b7-470c-a4e7-942a536e9627\") " pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.795202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795170 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-run\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795218 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-modprobe-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-lib-modules\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-conf\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795317 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-run\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-host\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795373 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-host\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795370 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-modprobe-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795385 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-d\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795389 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-systemd\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-sys\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-systemd\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-tmp\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-kubernetes\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795487 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysctl-conf\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysconfig\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795472 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-sys\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-lib-modules\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795545 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-sysconfig\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795554 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-var-lib-kubelet\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795557 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-kubernetes\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-tuned\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ae65159-0ebd-483a-8612-b9bf5b7b227f-var-lib-kubelet\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.795697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.795645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsfvv\" (UniqueName: \"kubernetes.io/projected/2ae65159-0ebd-483a-8612-b9bf5b7b227f-kube-api-access-dsfvv\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.797818 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.797790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-etc-tuned\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.798051 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.798032 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ae65159-0ebd-483a-8612-b9bf5b7b227f-tmp\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.803311 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.803289 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsfvv\" (UniqueName: \"kubernetes.io/projected/2ae65159-0ebd-483a-8612-b9bf5b7b227f-kube-api-access-dsfvv\") pod \"tuned-zjfn7\" (UID: \"2ae65159-0ebd-483a-8612-b9bf5b7b227f\") " pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:14.878221 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.878190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gvsbw" Jul 16 20:33:14.885193 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.885177 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:14.890853 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.890833 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-npv56" Jul 16 20:33:14.896980 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.896961 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5nssp" Jul 16 20:33:14.904645 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.904622 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-szbgs" Jul 16 20:33:14.910366 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.910350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grqmq" Jul 16 20:33:14.916984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.916964 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:14.923797 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.923782 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" Jul 16 20:33:14.928535 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:14.928517 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" Jul 16 20:33:15.197133 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.197105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:15.197289 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.197184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:15.197289 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.197256 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:15.197289 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.197269 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:15.197462 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.197312 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.197296329 +0000 UTC m=+4.039147435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:15.197462 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.197326 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.197320122 +0000 UTC m=+4.039171226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:15.297552 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.297519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:15.297741 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.297706 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:15.297741 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.297730 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:15.297741 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.297743 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:15.297898 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:15.297802 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.297783176 +0000 UTC m=+4.139634281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:15.399790 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.399761 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 20:33:15.422421 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.422395 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62974f8_89b7_470c_a4e7_942a536e9627.slice/crio-4b0243dfbdd2b831da36c93a7f4f82df0d7ad9b9e21535ffceaf5bb48c564ba1 WatchSource:0}: Error finding container 4b0243dfbdd2b831da36c93a7f4f82df0d7ad9b9e21535ffceaf5bb48c564ba1: Status 404 returned error can't find the container with id 4b0243dfbdd2b831da36c93a7f4f82df0d7ad9b9e21535ffceaf5bb48c564ba1 Jul 16 20:33:15.423706 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.423670 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b15667_eba3_4608_a2f7_2c955b9705dc.slice/crio-f0b401579d4312ab088417de5cce633942b45ad29042ccad3716414c1825d24a WatchSource:0}: Error finding container f0b401579d4312ab088417de5cce633942b45ad29042ccad3716414c1825d24a: Status 404 returned error can't find the container with id f0b401579d4312ab088417de5cce633942b45ad29042ccad3716414c1825d24a Jul 16 20:33:15.424432 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.424366 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d02010_e6f6_4108_9904_2e1e32aa50de.slice/crio-0e371bce3392bc4feec51e3b0a803e2c76028b6f257d885d80b15e0c3711585f WatchSource:0}: Error finding container 0e371bce3392bc4feec51e3b0a803e2c76028b6f257d885d80b15e0c3711585f: Status 404 returned error can't find the container with id 0e371bce3392bc4feec51e3b0a803e2c76028b6f257d885d80b15e0c3711585f Jul 16 20:33:15.427260 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.427229 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae65159_0ebd_483a_8612_b9bf5b7b227f.slice/crio-12a9df2599172fea70cc1659a601e2aa28102327039f0bfa19d4c4dbc530becc WatchSource:0}: Error finding container 12a9df2599172fea70cc1659a601e2aa28102327039f0bfa19d4c4dbc530becc: Status 404 returned error can't find the container with id 12a9df2599172fea70cc1659a601e2aa28102327039f0bfa19d4c4dbc530becc Jul 16 20:33:15.428312 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.428288 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe300f32_518a_4ff9_9700_d7ffd99864bd.slice/crio-19ebadd954642663ff158cad906987634d61a76722bbbc87d1619021705a32b4 WatchSource:0}: Error finding container 19ebadd954642663ff158cad906987634d61a76722bbbc87d1619021705a32b4: Status 404 returned error can't find the container with id 19ebadd954642663ff158cad906987634d61a76722bbbc87d1619021705a32b4 Jul 16 20:33:15.428624 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.428604 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d55b6a6_cbd1_41e1_97fc_80c64efeca7b.slice/crio-8ab7c388da110c8cf5f3b04eb9ffd755330c4ad0cad2579751247a452cc15d87 WatchSource:0}: Error finding container 8ab7c388da110c8cf5f3b04eb9ffd755330c4ad0cad2579751247a452cc15d87: Status 404 returned error can't find the container with id 8ab7c388da110c8cf5f3b04eb9ffd755330c4ad0cad2579751247a452cc15d87 Jul 16 20:33:15.429048 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.429001 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4ed8fd_86de_40e9_b633_280c9b429ec2.slice/crio-30f69fedc5bb7ff82aff65050fca437ff78131c71ce3b59ebbfdd1f4ae342bc0 WatchSource:0}: Error finding container 30f69fedc5bb7ff82aff65050fca437ff78131c71ce3b59ebbfdd1f4ae342bc0: Status 404 returned error can't find the container with id 30f69fedc5bb7ff82aff65050fca437ff78131c71ce3b59ebbfdd1f4ae342bc0 Jul 16 20:33:15.431018 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.430990 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6e688a_4008_41e9_970b_115e17aff958.slice/crio-edefd5bbe2464bb7ddc70b9390e4f2570fdbd60ee4fba8806a351cf08b169de9 WatchSource:0}: Error finding container edefd5bbe2464bb7ddc70b9390e4f2570fdbd60ee4fba8806a351cf08b169de9: Status 404 returned error can't find the container with id edefd5bbe2464bb7ddc70b9390e4f2570fdbd60ee4fba8806a351cf08b169de9 Jul 16 20:33:15.432296 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:15.432268 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8889d89f_ec52_4033_b2ca_58192f929616.slice/crio-a7bf485bec997373b6577301adeeec5b643dbb11768f883d565ed574701294fb WatchSource:0}: Error finding container a7bf485bec997373b6577301adeeec5b643dbb11768f883d565ed574701294fb: Status 404 returned error can't find the container with id a7bf485bec997373b6577301adeeec5b643dbb11768f883d565ed574701294fb Jul 16 20:33:15.621151 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.620958 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 20:28:14 +0000 UTC" deadline="2028-02-03 04:51:10.357670355 +0000 UTC" Jul 16 20:33:15.621613 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.621154 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13592h17m54.736521413s" Jul 16 20:33:15.724371 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.724340 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5nssp" event={"ID":"fe300f32-518a-4ff9-9700-d7ffd99864bd","Type":"ContainerStarted","Data":"19ebadd954642663ff158cad906987634d61a76722bbbc87d1619021705a32b4"} Jul 16 20:33:15.725376 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.725343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gvsbw" event={"ID":"b9b15667-eba3-4608-a2f7-2c955b9705dc","Type":"ContainerStarted","Data":"f0b401579d4312ab088417de5cce633942b45ad29042ccad3716414c1825d24a"} Jul 16 20:33:15.727063 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.727038 2582 generic.go:356] "Generic (PLEG): container finished" podID="64bfe31dea644c334a88e35254b8a1e2" containerID="826f0fe5f28bd16129538a8eb079d446f84699b70ca982c7f2cf6eb6fa486247" exitCode=0 Jul 16 20:33:15.727138 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.727119 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"64bfe31dea644c334a88e35254b8a1e2","Type":"ContainerDied","Data":"826f0fe5f28bd16129538a8eb079d446f84699b70ca982c7f2cf6eb6fa486247"} Jul 16 20:33:15.728788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.728741 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"9f3a70c0e179a36534f61d6399c4980f","Type":"ContainerStarted","Data":"02780d26b5b7f9b77193d88fad8ca4a9623dd6e411457fb4209b13b7af09f177"} Jul 16 20:33:15.729966 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.729946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" event={"ID":"7c6e688a-4008-41e9-970b-115e17aff958","Type":"ContainerStarted","Data":"edefd5bbe2464bb7ddc70b9390e4f2570fdbd60ee4fba8806a351cf08b169de9"} Jul 16 20:33:15.730971 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.730945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgj47" event={"ID":"3a4ed8fd-86de-40e9-b633-280c9b429ec2","Type":"ContainerStarted","Data":"30f69fedc5bb7ff82aff65050fca437ff78131c71ce3b59ebbfdd1f4ae342bc0"} Jul 16 20:33:15.732060 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.732042 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" event={"ID":"2ae65159-0ebd-483a-8612-b9bf5b7b227f","Type":"ContainerStarted","Data":"12a9df2599172fea70cc1659a601e2aa28102327039f0bfa19d4c4dbc530becc"} Jul 16 20:33:15.733115 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.733092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerStarted","Data":"0e371bce3392bc4feec51e3b0a803e2c76028b6f257d885d80b15e0c3711585f"} Jul 16 20:33:15.734206 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.734187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grqmq" event={"ID":"a62974f8-89b7-470c-a4e7-942a536e9627","Type":"ContainerStarted","Data":"4b0243dfbdd2b831da36c93a7f4f82df0d7ad9b9e21535ffceaf5bb48c564ba1"} Jul 16 20:33:15.735305 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.735285 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"a7bf485bec997373b6577301adeeec5b643dbb11768f883d565ed574701294fb"} Jul 16 20:33:15.736292 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.736272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-npv56" event={"ID":"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b","Type":"ContainerStarted","Data":"8ab7c388da110c8cf5f3b04eb9ffd755330c4ad0cad2579751247a452cc15d87"} Jul 16 20:33:15.757456 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:15.757415 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" podStartSLOduration=2.757400288 podStartE2EDuration="2.757400288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:15.757367128 +0000 UTC m=+3.599218256" watchObservedRunningTime="2026-07-16 20:33:15.757400288 +0000 UTC m=+3.599251421" Jul 16 20:33:16.204112 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.203968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:16.204112 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.204026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:16.204318 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.204146 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:16.204318 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.204203 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:18.204185041 +0000 UTC m=+6.046036145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:16.204628 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.204611 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:16.204706 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.204667 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:18.204652461 +0000 UTC m=+6.046503565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:16.304601 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.304556 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:16.304779 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.304761 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:16.304846 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.304794 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:16.304846 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.304807 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:16.304949 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.304856 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:18.30484039 +0000 UTC m=+6.146691509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:16.622463 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.621444 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 20:28:14 +0000 UTC" deadline="2028-02-23 07:37:02.848980793 +0000 UTC" Jul 16 20:33:16.622463 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.621483 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14075h3m46.227502865s" Jul 16 20:33:16.717828 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.717713 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:16.717975 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.717858 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:16.717975 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.717937 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:16.718041 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.718004 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:16.718071 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.718056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:16.718135 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:16.718114 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:16.771729 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:16.771689 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"64bfe31dea644c334a88e35254b8a1e2","Type":"ContainerStarted","Data":"0d4199a8e9e4d71b19723dc605cd3e7a4b1d723cb354909f4a1828b05d77b605"} Jul 16 20:33:18.220964 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.220930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:18.221430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.220989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:18.221430 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.221128 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:18.221430 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.221211 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:22.221174999 +0000 UTC m=+10.063026106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:18.221621 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.221597 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:18.221686 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.221647 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:22.221632607 +0000 UTC m=+10.063483712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:18.321648 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.321615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:18.321826 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.321759 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:18.321826 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.321779 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:18.321826 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.321791 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:18.322110 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.321842 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:22.321825594 +0000 UTC m=+10.163676703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:18.718513 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.717656 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:18.718513 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.717816 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:18.718513 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.718213 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:18.718513 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.718335 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:18.719006 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:18.718870 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:18.719006 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:18.718973 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:20.716884 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:20.716816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:20.717321 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:20.716960 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:20.717382 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:20.717373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:20.717507 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:20.717475 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:20.717636 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:20.717620 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:20.717728 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:20.717708 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:22.253545 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.253509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:22.253997 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.253586 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:22.253997 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.253732 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:22.253997 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.253789 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:30.253770007 +0000 UTC m=+18.095621129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:22.254198 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.254182 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:22.254250 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.254232 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:30.254217118 +0000 UTC m=+18.096068228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:22.354938 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.354906 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:22.355115 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.355073 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:22.355115 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.355092 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:22.355115 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.355105 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:22.355270 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.355154 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:30.355140144 +0000 UTC m=+18.196991249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:22.717956 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.717380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:22.717956 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.717497 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:22.717956 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.717868 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:22.718217 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.717955 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:22.718217 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:22.718003 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:22.718217 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:22.718098 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:24.717328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:24.717300 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:24.717328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:24.717328 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:24.717807 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:24.717300 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:24.717807 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:24.717409 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:24.717807 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:24.717506 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:24.717807 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:24.717566 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:26.716939 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:26.716908 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:26.716939 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:26.716945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:26.717506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:26.716908 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:26.717506 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:26.717027 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:26.717506 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:26.717086 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:26.717506 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:26.717165 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:28.717275 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:28.717240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:28.717760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:28.717254 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:28.717760 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:28.717364 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:28.717760 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:28.717457 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:28.717760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:28.717485 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:28.717760 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:28.717613 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:30.320168 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.320136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:30.320642 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.320210 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:30.320642 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.320299 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:30.320642 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.320318 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:30.320642 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.320377 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:46.320354682 +0000 UTC m=+34.162205787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:30.320642 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.320391 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:33:46.320384838 +0000 UTC m=+34.162235943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:30.420957 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.420931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:30.421123 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.421045 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:30.421123 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.421062 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:30.421123 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.421075 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:30.421262 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.421126 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:33:46.421110942 +0000 UTC m=+34.262962048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:30.717556 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.717479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:30.717737 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.717479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:30.717737 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:30.717624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:30.717737 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.717650 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:30.717737 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.717698 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:30.717936 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:30.717750 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:32.717414 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:32.717381 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:32.717837 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:32.717507 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:32.717837 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:32.717555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:32.717837 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:32.717634 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:32.717837 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:32.717662 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:32.717837 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:32.717759 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:34.717419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.717117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:34.718116 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.717118 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:34.718116 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:34.717511 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:34.718116 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.717195 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:34.718116 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:34.717607 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:34.718116 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:34.717666 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:34.807207 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807184 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:33:34.807528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807507 2582 generic.go:356] "Generic (PLEG): container finished" podID="8889d89f-ec52-4033-b2ca-58192f929616" containerID="49fdc66a94cd5277f07208c670ce074f69fc130ef08f9b4d84b5be59c66b43b5" exitCode=1 Jul 16 20:33:34.807646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807594 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"80b7a5adf4900d9dc05a4a320f6648fa7d68e978be107e52b062d781e3d4edec"} Jul 16 20:33:34.807706 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"d98752bdc1be293118ae8c68d2a1b68aab5585fcc016e4648e1827bc474c306b"} Jul 16 20:33:34.807756 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"d14460b78715061902afdde191e04412d13c676595f7a6513afcef5493465fc0"} Jul 16 20:33:34.807756 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807737 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"9ab39be0d7df7babd454dda821b3de383cbd29fbe1118ca374e3b55a088af089"} Jul 16 20:33:34.807839 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerDied","Data":"49fdc66a94cd5277f07208c670ce074f69fc130ef08f9b4d84b5be59c66b43b5"} Jul 16 20:33:34.807839 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.807801 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"76ceece0c81801101057343f29dc6b70141107aaf648fe07759baf4ab7ff6bfa"} Jul 16 20:33:34.808907 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.808880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-npv56" event={"ID":"5d55b6a6-cbd1-41e1-97fc-80c64efeca7b","Type":"ContainerStarted","Data":"987f980519c23cc7ee8c71446c1e1383354df66b8188bd2f802b3d4c53eba55e"} Jul 16 20:33:34.810521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.810459 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5nssp" event={"ID":"fe300f32-518a-4ff9-9700-d7ffd99864bd","Type":"ContainerStarted","Data":"b883bf8abbec5c0295d665373950c44ecff256d3fe52ef13b98b8d4ae8f70952"} Jul 16 20:33:34.812076 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.812041 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gvsbw" event={"ID":"b9b15667-eba3-4608-a2f7-2c955b9705dc","Type":"ContainerStarted","Data":"68136c60cf7c0f93cd03b9993570f499058edbd97fb453c39e0c9638d9696576"} Jul 16 20:33:34.813624 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.813567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" event={"ID":"7c6e688a-4008-41e9-970b-115e17aff958","Type":"ContainerStarted","Data":"3a8d11bf1aed8f2c287f9516b17cc84f105fb0ba3517c4db14f7e8b43fe709f4"} Jul 16 20:33:34.814993 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.814973 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgj47" event={"ID":"3a4ed8fd-86de-40e9-b633-280c9b429ec2","Type":"ContainerStarted","Data":"482c89ffe6a8502ae0cb695d8a984a8b6caa39ca3aa979d5b9058469e9eb9afe"} Jul 16 20:33:34.816464 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.816443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" event={"ID":"2ae65159-0ebd-483a-8612-b9bf5b7b227f","Type":"ContainerStarted","Data":"d5a24cd9ee34a3606b9f73de98ccfe07610cc6689940efab4a1ed5d71d54af77"} Jul 16 20:33:34.818023 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.818005 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="917977608b8bc17aaa44503f79f981099221a4744d450fab6bdd2cff947e0df7" exitCode=0 Jul 16 20:33:34.818130 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.818033 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"917977608b8bc17aaa44503f79f981099221a4744d450fab6bdd2cff947e0df7"} Jul 16 20:33:34.823121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.823081 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" podStartSLOduration=21.823069574 podStartE2EDuration="21.823069574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:16.788125371 +0000 UTC m=+4.629976501" watchObservedRunningTime="2026-07-16 20:33:34.823069574 +0000 UTC m=+22.664920692" Jul 16 20:33:34.823336 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.823312 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-npv56" podStartSLOduration=4.41906249 podStartE2EDuration="22.823305591s" totalImagesPullingTime="18.404243101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:34.822617776 +0000 UTC m=+22.664468901" watchObservedRunningTime="2026-07-16 20:33:34.823305591 +0000 UTC m=+22.665156716" Jul 16 20:33:34.860017 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.859977 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zjfn7" podStartSLOduration=4.453458126 podStartE2EDuration="22.859965483s" totalImagesPullingTime="18.406507357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:34.859617847 +0000 UTC m=+22.701468986" watchObservedRunningTime="2026-07-16 20:33:34.859965483 +0000 UTC m=+22.701816678" Jul 16 20:33:34.873429 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.873388 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gvsbw" podStartSLOduration=4.464830335 podStartE2EDuration="22.873374041s" totalImagesPullingTime="18.408543706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:34.873032279 +0000 UTC m=+22.714883415" watchObservedRunningTime="2026-07-16 20:33:34.873374041 +0000 UTC m=+22.715225168" Jul 16 20:33:34.900487 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.900415 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wgj47" podStartSLOduration=9.078199523 podStartE2EDuration="22.900403688s" totalImagesPullingTime="13.822204165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:34.900397212 +0000 UTC m=+22.742248338" watchObservedRunningTime="2026-07-16 20:33:34.900403688 +0000 UTC m=+22.742254814" Jul 16 20:33:34.922285 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:34.922245 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5nssp" podStartSLOduration=3.971019566 podStartE2EDuration="22.922231514s" totalImagesPullingTime="18.951211948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:34.922070616 +0000 UTC m=+22.763921745" watchObservedRunningTime="2026-07-16 20:33:34.922231514 +0000 UTC m=+22.764082648" Jul 16 20:33:35.481813 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.481793 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 20:33:35.654236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.654108 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T20:33:35.481809913Z","UUID":"3658c49f-9f8b-481a-af8d-3628bec8deba","Handler":null,"Name":"","Endpoint":""} Jul 16 20:33:35.656000 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.655973 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 20:33:35.656111 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.656017 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 20:33:35.821402 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.821370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" event={"ID":"7c6e688a-4008-41e9-970b-115e17aff958","Type":"ContainerStarted","Data":"f08633cf163e3ba5d2269029aac5b0bab7210618e535277de67a63fdade14287"} Jul 16 20:33:35.822796 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.822766 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grqmq" event={"ID":"a62974f8-89b7-470c-a4e7-942a536e9627","Type":"ContainerStarted","Data":"c9d9b6d701a2885398d978f6d3e51dc0b13697b7d0efb1d634d992a5338de114"} Jul 16 20:33:35.835327 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:35.835290 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-grqmq" podStartSLOduration=5.425342952 podStartE2EDuration="23.835277582s" totalImagesPullingTime="18.40993463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:35.835129162 +0000 UTC m=+23.676980290" watchObservedRunningTime="2026-07-16 20:33:35.835277582 +0000 UTC m=+23.677128707" Jul 16 20:33:36.717419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.717380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:36.717733 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:36.717524 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:36.717733 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.717380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:36.717733 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.717599 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:36.717733 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:36.717685 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:36.717943 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:36.717764 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:36.827648 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.827615 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" event={"ID":"7c6e688a-4008-41e9-970b-115e17aff958","Type":"ContainerStarted","Data":"5aeff680a1f928fb7866ececc8fb7325d156443f9dfae5098fb48614e931ab4d"} Jul 16 20:33:36.831384 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.831360 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:33:36.831775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.831745 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"9f09b80cb49b0c9d12ad36331bb30c10a26384d51a1972f0505c6a80f77d5f4c"} Jul 16 20:33:36.845745 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:36.845708 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44g67" podStartSLOduration=4.283664031 podStartE2EDuration="24.845693533s" totalImagesPullingTime="20.562029502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:36.845256168 +0000 UTC m=+24.687107295" watchObservedRunningTime="2026-07-16 20:33:36.845693533 +0000 UTC m=+24.687544663" Jul 16 20:33:37.445204 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:37.445151 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:37.445820 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:37.445798 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:37.835455 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:37.835313 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:37.835958 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:37.835931 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wgj47" Jul 16 20:33:38.720298 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:38.720274 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:38.720498 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:38.720365 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:38.720498 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:38.720377 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:38.720498 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:38.720396 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:38.720498 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:38.720453 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:38.720690 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:38.720522 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:39.843239 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.843180 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:33:39.844995 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.844974 2582 scope.go:117] "RemoveContainer" containerID="49fdc66a94cd5277f07208c670ce074f69fc130ef08f9b4d84b5be59c66b43b5" Jul 16 20:33:39.845266 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.845239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"74deae5b65fe0a9a085299736b120e928e6ee1a7c50058c1e72a3082fa4dfbfd"} Jul 16 20:33:39.845379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.845360 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:39.845518 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.845504 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:39.845518 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.845555 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:39.861209 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.861028 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:39.861609 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:39.861566 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:33:40.717067 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.717007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:40.717067 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.717038 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:40.717294 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.717014 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:40.717294 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:40.717098 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:40.717294 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:40.717200 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:40.717294 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:40.717257 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:40.847446 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.847421 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="782b083e646dba14ea64d752ae6d78658d5150ab2a26c7e0fff4b89aeb4ae283" exitCode=0 Jul 16 20:33:40.847837 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.847454 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"782b083e646dba14ea64d752ae6d78658d5150ab2a26c7e0fff4b89aeb4ae283"} Jul 16 20:33:40.853565 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.853355 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:33:40.857392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.857361 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" event={"ID":"8889d89f-ec52-4033-b2ca-58192f929616","Type":"ContainerStarted","Data":"560b6b993e212d33e01ab36c215b73a8ddd4a9289b42a06f330a92ed3bccd088"} Jul 16 20:33:40.931039 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:40.930985 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" podStartSLOduration=10.188511836 podStartE2EDuration="28.930965801s" totalImagesPullingTime="18.742453965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:40.930381429 +0000 UTC m=+28.772232597" watchObservedRunningTime="2026-07-16 20:33:40.930965801 +0000 UTC m=+28.772816927" Jul 16 20:33:41.644997 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.644969 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pjcxx"] Jul 16 20:33:41.645204 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.645070 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:41.645204 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:41.645153 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:41.646644 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.646613 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84n8n"] Jul 16 20:33:41.646737 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.646727 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:41.646832 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:41.646810 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:41.648279 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.648258 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9sqrn"] Jul 16 20:33:41.648356 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:41.648346 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:41.648459 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:41.648442 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:42.862986 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:42.862959 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="61e4fbf16dc3abeec2181d124bdd872e6ab266361b1d4512b58ae88f910d2408" exitCode=0 Jul 16 20:33:42.863324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:42.863013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"61e4fbf16dc3abeec2181d124bdd872e6ab266361b1d4512b58ae88f910d2408"} Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:43.716558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:43.716869 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:43.716602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:43.716970 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:43.716591 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:43.717410 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:43.717035 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:43.867116 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:43.867090 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="c11fdd432ed105f3d99769461f7082c30e7303b4cf4d0d45d9c9e12cf3a75976" exitCode=0 Jul 16 20:33:43.867416 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:43.867141 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"c11fdd432ed105f3d99769461f7082c30e7303b4cf4d0d45d9c9e12cf3a75976"} Jul 16 20:33:45.716775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:45.716737 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:45.717189 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:45.716854 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:45.717189 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:45.716865 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:45.717189 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:45.716863 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjcxx" podUID="a52312b6-20e0-4f0f-847b-a9743c34320b" Jul 16 20:33:45.717189 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:45.716978 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9sqrn" podUID="c9238df2-51ed-4a8b-870b-15cf82c175df" Jul 16 20:33:45.717189 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:45.717053 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n8n" podUID="5556d69e-b310-4298-888e-5f47489a878b" Jul 16 20:33:46.336753 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:46.336712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:46.336921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:46.336788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:46.336921 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.336887 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:46.337009 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.336942 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret podName:5556d69e-b310-4298-888e-5f47489a878b nodeName:}" failed. No retries permitted until 2026-07-16 20:34:18.336929161 +0000 UTC m=+66.178780266 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret") pod "global-pull-secret-syncer-84n8n" (UID: "5556d69e-b310-4298-888e-5f47489a878b") : object "kube-system"/"original-pull-secret" not registered Jul 16 20:33:46.337009 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.336888 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:46.337009 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.336975 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs podName:c9238df2-51ed-4a8b-870b-15cf82c175df nodeName:}" failed. No retries permitted until 2026-07-16 20:34:18.336968358 +0000 UTC m=+66.178819463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs") pod "network-metrics-daemon-9sqrn" (UID: "c9238df2-51ed-4a8b-870b-15cf82c175df") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 20:33:46.437352 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:46.437321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:46.437534 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.437506 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 20:33:46.437534 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.437536 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 20:33:46.437761 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.437551 2582 projected.go:196] Error preparing data for projected volume kube-api-access-g5n59 for pod openshift-network-diagnostics/network-check-target-pjcxx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:46.437761 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:46.437624 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59 podName:a52312b6-20e0-4f0f-847b-a9743c34320b nodeName:}" failed. No retries permitted until 2026-07-16 20:34:18.437605399 +0000 UTC m=+66.279456504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g5n59" (UniqueName: "kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59") pod "network-check-target-pjcxx" (UID: "a52312b6-20e0-4f0f-847b-a9743c34320b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 20:33:47.012138 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.012100 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeReady" Jul 16 20:33:47.012627 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.012265 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 20:33:47.060530 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.060498 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw"] Jul 16 20:33:47.086639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.086519 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:33:47.087294 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.087268 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.091119 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.091095 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 20:33:47.091435 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.091248 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.091435 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.091351 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.091603 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.091435 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 20:33:47.091603 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.091549 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lpmqd\"" Jul 16 20:33:47.101205 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.100453 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k7vcw"] Jul 16 20:33:47.101205 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.100700 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.104338 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.103656 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 20:33:47.105801 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.105564 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 20:33:47.106118 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.106098 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 20:33:47.106450 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.106423 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v7sr7\"" Jul 16 20:33:47.116884 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.116858 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 20:33:47.122712 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.122682 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs"] Jul 16 20:33:47.122843 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.122826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.129791 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.129767 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 20:33:47.130061 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.130046 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.130128 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.130074 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.130287 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.130267 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rzq4b\"" Jul 16 20:33:47.130591 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.130553 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 20:33:47.136675 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.136659 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 20:33:47.144619 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.144597 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979"] Jul 16 20:33:47.144703 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.144658 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.147448 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.147425 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tdjft\"" Jul 16 20:33:47.148178 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.148157 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 20:33:47.148262 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.148178 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.148262 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.148248 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.148373 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.148264 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 20:33:47.162724 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.162701 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fdc7s"] Jul 16 20:33:47.162836 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.162816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.165265 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.165249 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 20:33:47.165479 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.165458 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.165560 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.165517 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.165780 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.165761 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dkwdv\"" Jul 16 20:33:47.183271 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.183248 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp"] Jul 16 20:33:47.183437 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.183420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.187562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.187530 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 20:33:47.187669 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.187603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 20:33:47.187669 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.187634 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.187669 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.187652 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.188162 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.188143 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lsn74\"" Jul 16 20:33:47.197909 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.197889 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 20:33:47.199936 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.199913 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-665df85b66-nl88f"] Jul 16 20:33:47.200051 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.200032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.202795 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.202774 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 20:33:47.202999 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.202981 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.203085 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.203019 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 20:33:47.203085 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.203024 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.203190 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.203156 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dv94z\"" Jul 16 20:33:47.212364 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.212348 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh"] Jul 16 20:33:47.212486 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.212469 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.218120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218090 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.218201 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218184 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.218259 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218203 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 20:33:47.218802 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218784 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-s5gxc\"" Jul 16 20:33:47.218802 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218797 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 20:33:47.218952 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.218789 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 20:33:47.219261 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.219244 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 20:33:47.230434 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230418 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fdc7s"] Jul 16 20:33:47.230527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230443 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs"] Jul 16 20:33:47.230527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230455 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw"] Jul 16 20:33:47.230527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230468 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979"] Jul 16 20:33:47.230527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230493 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2z2lm"] Jul 16 20:33:47.230743 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.230544 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" Jul 16 20:33:47.234318 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.234302 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hr4fl\"" Jul 16 20:33:47.234535 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.234521 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.234629 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.234558 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.241646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/640a485f-3383-44c9-b7ff-1974fe02179e-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.241736 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.241736 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241673 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.241736 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.241736 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7vzs\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfb6b\" (UniqueName: \"kubernetes.io/projected/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-kube-api-access-mfb6b\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241857 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-snapshots\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpfk\" (UniqueName: \"kubernetes.io/projected/da4a5426-e1fa-4a86-886b-8415a2614e7e-kube-api-access-vcpfk\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.241912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241905 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nwmg\" (UniqueName: \"kubernetes.io/projected/640a485f-3383-44c9-b7ff-1974fe02179e-kube-api-access-4nwmg\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/640a485f-3383-44c9-b7ff-1974fe02179e-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.241998 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-922sz\" (UniqueName: \"kubernetes.io/projected/8fbde4a0-208d-4fd8-abfa-d9883ad48241-kube-api-access-922sz\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242053 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242069 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.242121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242104 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fbde4a0-208d-4fd8-abfa-d9883ad48241-serving-cert\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.242385 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fbde4a0-208d-4fd8-abfa-d9883ad48241-config\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.242385 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da4a5426-e1fa-4a86-886b-8415a2614e7e-serving-cert\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.242385 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242224 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-tmp\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.242385 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.242249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.256532 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.256511 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5"] Jul 16 20:33:47.256849 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.256691 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.260002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.259746 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.260002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.259774 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 20:33:47.260002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.259806 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.260002 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.259848 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c96n9\"" Jul 16 20:33:47.276777 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.276757 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm"] Jul 16 20:33:47.276871 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.276848 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.279645 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.279627 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 20:33:47.279748 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.279626 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j5fc2\"" Jul 16 20:33:47.279748 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.279714 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 20:33:47.298828 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298811 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:33:47.298932 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-665df85b66-nl88f"] Jul 16 20:33:47.298932 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298855 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh"] Jul 16 20:33:47.298932 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298868 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp"] Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298934 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298962 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5"] Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298979 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2z2lm"] Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.298993 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k7vcw"] Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.299009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm"] Jul 16 20:33:47.299088 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.299031 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x958g"] Jul 16 20:33:47.301417 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.301400 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 20:33:47.301566 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.301546 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 20:33:47.301705 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.301486 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7xpwr\"" Jul 16 20:33:47.316906 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.316857 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x958g"] Jul 16 20:33:47.316994 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.316961 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.319779 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.319761 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 20:33:47.320059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.320039 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q8qkr\"" Jul 16 20:33:47.320275 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.320260 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 20:33:47.342980 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.342947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpfk\" (UniqueName: \"kubernetes.io/projected/da4a5426-e1fa-4a86-886b-8415a2614e7e-kube-api-access-vcpfk\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.343083 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.342980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76fh6\" (UniqueName: \"kubernetes.io/projected/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-kube-api-access-76fh6\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.343140 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.343202 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.343253 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.343253 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343236 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41e2fac7-5f11-405f-aa74-f4241008bf24-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.343348 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343265 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fbde4a0-208d-4fd8-abfa-d9883ad48241-serving-cert\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.343348 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.343348 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343336 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-stats-auth\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.343478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343365 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.343478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343394 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35dd9ae8-1456-44ff-8b64-5790858c897c-serving-cert\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.343478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/640a485f-3383-44c9-b7ff-1974fe02179e-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.343478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.343478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7vzs\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m55hd\" (UniqueName: \"kubernetes.io/projected/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-kube-api-access-m55hd\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-cert\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw2kr\" (UniqueName: \"kubernetes.io/projected/3d410c6a-51fe-4d68-bc58-d72bbca0e752-kube-api-access-gw2kr\") pod \"volume-data-source-validator-78cf4c6f6-qcqxh\" (UID: \"3d410c6a-51fe-4d68-bc58-d72bbca0e752\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b78c5\" (UniqueName: \"kubernetes.io/projected/41e2fac7-5f11-405f-aa74-f4241008bf24-kube-api-access-b78c5\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nwmg\" (UniqueName: \"kubernetes.io/projected/640a485f-3383-44c9-b7ff-1974fe02179e-kube-api-access-4nwmg\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.343699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-service-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.343984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343704 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-config\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.343984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-trusted-ca\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.343984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.343984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.343798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/640a485f-3383-44c9-b7ff-1974fe02179e-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.344401 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/640a485f-3383-44c9-b7ff-1974fe02179e-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.344521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344504 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da4a5426-e1fa-4a86-886b-8415a2614e7e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.344606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.344656 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-922sz\" (UniqueName: \"kubernetes.io/projected/8fbde4a0-208d-4fd8-abfa-d9883ad48241-kube-api-access-922sz\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.344656 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da4a5426-e1fa-4a86-886b-8415a2614e7e-serving-cert\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.344821 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.344674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fbde4a0-208d-4fd8-abfa-d9883ad48241-config\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.344938 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.344920 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:33:47.344999 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.344940 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:33:47.344999 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.344991 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.844973111 +0000 UTC m=+35.686824232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345190 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fbde4a0-208d-4fd8-abfa-d9883ad48241-config\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-tmp\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345275 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.345399 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.345503 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.845487212 +0000 UTC m=+35.687338317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345534 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345563 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-tmp\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345611 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfb6b\" (UniqueName: \"kubernetes.io/projected/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-kube-api-access-mfb6b\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.345639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345642 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-default-certificate\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.346187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7nn\" (UniqueName: \"kubernetes.io/projected/35dd9ae8-1456-44ff-8b64-5790858c897c-kube-api-access-vf7nn\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.346187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.345726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-snapshots\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.346326 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.346305 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da4a5426-e1fa-4a86-886b-8415a2614e7e-snapshots\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.348465 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.348390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da4a5426-e1fa-4a86-886b-8415a2614e7e-serving-cert\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.348465 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.348450 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fbde4a0-208d-4fd8-abfa-d9883ad48241-serving-cert\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.348771 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.348749 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/640a485f-3383-44c9-b7ff-1974fe02179e-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.352928 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.352885 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.353185 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.353161 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.353280 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.353210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.353399 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.353374 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.353496 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.353374 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.356843 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.356749 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-922sz\" (UniqueName: \"kubernetes.io/projected/8fbde4a0-208d-4fd8-abfa-d9883ad48241-kube-api-access-922sz\") pod \"service-ca-operator-5bcbc94c78-x29cw\" (UID: \"8fbde4a0-208d-4fd8-abfa-d9883ad48241\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.357293 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.357236 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.357853 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.357830 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfb6b\" (UniqueName: \"kubernetes.io/projected/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-kube-api-access-mfb6b\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.358542 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.358444 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nwmg\" (UniqueName: \"kubernetes.io/projected/640a485f-3383-44c9-b7ff-1974fe02179e-kube-api-access-4nwmg\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsgfs\" (UID: \"640a485f-3383-44c9-b7ff-1974fe02179e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.358751 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.358723 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpfk\" (UniqueName: \"kubernetes.io/projected/da4a5426-e1fa-4a86-886b-8415a2614e7e-kube-api-access-vcpfk\") pod \"insights-operator-555d9d7885-k7vcw\" (UID: \"da4a5426-e1fa-4a86-886b-8415a2614e7e\") " pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.359324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.359282 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7vzs\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.400179 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.400153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" Jul 16 20:33:47.432660 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.432642 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" Jul 16 20:33:47.446424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-stats-auth\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.446506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.446506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35dd9ae8-1456-44ff-8b64-5790858c897c-serving-cert\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.446506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m55hd\" (UniqueName: \"kubernetes.io/projected/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-kube-api-access-m55hd\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.446666 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-cert\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.446666 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw2kr\" (UniqueName: \"kubernetes.io/projected/3d410c6a-51fe-4d68-bc58-d72bbca0e752-kube-api-access-gw2kr\") pod \"volume-data-source-validator-78cf4c6f6-qcqxh\" (UID: \"3d410c6a-51fe-4d68-bc58-d72bbca0e752\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" Jul 16 20:33:47.446666 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78c5\" (UniqueName: \"kubernetes.io/projected/41e2fac7-5f11-405f-aa74-f4241008bf24-kube-api-access-b78c5\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.446666 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.446624 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.946600689 +0000 UTC m=+35.788451814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:33:47.446857 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-config\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.446857 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-trusted-ca\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.446857 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446738 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.446857 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.446792 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc407f8-7328-46c2-bbf9-53e6bae72070-tmp-dir\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.447055 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.446927 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:47.447055 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.446965 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.946952938 +0000 UTC m=+35.788804056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:47.447160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.447160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.447160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-default-certificate\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.447302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7nn\" (UniqueName: \"kubernetes.io/projected/35dd9ae8-1456-44ff-8b64-5790858c897c-kube-api-access-vf7nn\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.447302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fc407f8-7328-46c2-bbf9-53e6bae72070-config-volume\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.447302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76fh6\" (UniqueName: \"kubernetes.io/projected/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-kube-api-access-76fh6\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.447302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29dr\" (UniqueName: \"kubernetes.io/projected/74a84d24-de40-4cc1-91ac-a3ff3d003c2e-kube-api-access-n29dr\") pod \"network-check-source-7bc5b6bf85-stvbm\" (UID: \"74a84d24-de40-4cc1-91ac-a3ff3d003c2e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" Jul 16 20:33:47.447480 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.447480 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.447480 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447380 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtcd6\" (UniqueName: \"kubernetes.io/projected/4fc407f8-7328-46c2-bbf9-53e6bae72070-kube-api-access-mtcd6\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.447480 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41e2fac7-5f11-405f-aa74-f4241008bf24-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.447698 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.447182 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:33:47.447698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.447514 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-config\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.447698 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.447531 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.947516309 +0000 UTC m=+35.789367419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:33:47.448383 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.448284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35dd9ae8-1456-44ff-8b64-5790858c897c-trusted-ca\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.449400 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.449359 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41e2fac7-5f11-405f-aa74-f4241008bf24-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.449400 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.449370 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-stats-auth\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.449540 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.449422 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35dd9ae8-1456-44ff-8b64-5790858c897c-serving-cert\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.449623 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.449589 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-cert\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.449996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.449978 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-default-certificate\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.453699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.453542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" Jul 16 20:33:47.455433 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.455390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76fh6\" (UniqueName: \"kubernetes.io/projected/e6eb2d8f-ddfe-403b-91ea-75cecba86f98-kube-api-access-76fh6\") pod \"ingress-canary-2z2lm\" (UID: \"e6eb2d8f-ddfe-403b-91ea-75cecba86f98\") " pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.455903 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.455858 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7nn\" (UniqueName: \"kubernetes.io/projected/35dd9ae8-1456-44ff-8b64-5790858c897c-kube-api-access-vf7nn\") pod \"console-operator-565b7b569d-fdc7s\" (UID: \"35dd9ae8-1456-44ff-8b64-5790858c897c\") " pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.456495 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.456412 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw2kr\" (UniqueName: \"kubernetes.io/projected/3d410c6a-51fe-4d68-bc58-d72bbca0e752-kube-api-access-gw2kr\") pod \"volume-data-source-validator-78cf4c6f6-qcqxh\" (UID: \"3d410c6a-51fe-4d68-bc58-d72bbca0e752\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" Jul 16 20:33:47.457098 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.457079 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m55hd\" (UniqueName: \"kubernetes.io/projected/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-kube-api-access-m55hd\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.457171 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.457098 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b78c5\" (UniqueName: \"kubernetes.io/projected/41e2fac7-5f11-405f-aa74-f4241008bf24-kube-api-access-b78c5\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.497057 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.497028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:47.540082 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.540051 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" Jul 16 20:33:47.548392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc407f8-7328-46c2-bbf9-53e6bae72070-tmp-dir\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.548493 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.548493 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fc407f8-7328-46c2-bbf9-53e6bae72070-config-volume\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.548606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29dr\" (UniqueName: \"kubernetes.io/projected/74a84d24-de40-4cc1-91ac-a3ff3d003c2e-kube-api-access-n29dr\") pod \"network-check-source-7bc5b6bf85-stvbm\" (UID: \"74a84d24-de40-4cc1-91ac-a3ff3d003c2e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" Jul 16 20:33:47.548606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.548606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.548606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtcd6\" (UniqueName: \"kubernetes.io/projected/4fc407f8-7328-46c2-bbf9-53e6bae72070-kube-api-access-mtcd6\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.548788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.548653 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:33:47.548788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.548730 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.048708135 +0000 UTC m=+35.890559260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:33:47.548788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.548752 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc407f8-7328-46c2-bbf9-53e6bae72070-tmp-dir\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.548919 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.548862 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:33:47.548964 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.548952 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.048934847 +0000 UTC m=+35.890785967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:33:47.549060 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.549041 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fc407f8-7328-46c2-bbf9-53e6bae72070-config-volume\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.549373 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.549354 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:47.557691 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.557670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtcd6\" (UniqueName: \"kubernetes.io/projected/4fc407f8-7328-46c2-bbf9-53e6bae72070-kube-api-access-mtcd6\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:47.557770 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.557759 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29dr\" (UniqueName: \"kubernetes.io/projected/74a84d24-de40-4cc1-91ac-a3ff3d003c2e-kube-api-access-n29dr\") pod \"network-check-source-7bc5b6bf85-stvbm\" (UID: \"74a84d24-de40-4cc1-91ac-a3ff3d003c2e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" Jul 16 20:33:47.566453 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.566432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2z2lm" Jul 16 20:33:47.612187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.612162 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" Jul 16 20:33:47.716927 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.716896 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:33:47.716927 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.716926 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:33:47.717175 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.716926 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:33:47.719872 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.719733 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 20:33:47.719872 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.719745 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 20:33:47.719872 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.719746 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ss679\"" Jul 16 20:33:47.719872 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.719756 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xxdx\"" Jul 16 20:33:47.850773 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.850697 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:47.850906 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.850840 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:33:47.850906 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.850863 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:33:47.850906 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.850874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:47.851060 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.850921 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.85090486 +0000 UTC m=+36.692755969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:33:47.851060 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.850969 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:33:47.851060 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.851029 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.851013609 +0000 UTC m=+36.692864728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:33:47.951443 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.951413 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.951624 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.951503 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:47.951624 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.951551 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:33:47.951744 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:47.951560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:47.951744 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.951632 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.951612615 +0000 UTC m=+36.793463720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:33:47.951744 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.951653 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:47.951744 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.951660 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.95164949 +0000 UTC m=+36.793500601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:33:47.951744 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:47.951709 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:48.951694141 +0000 UTC m=+36.793545251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:48.052403 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.052371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:48.052980 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.052526 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:33:48.052980 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.052546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:48.052980 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.052606 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:49.052588277 +0000 UTC m=+36.894439395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:33:48.052980 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.052649 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:33:48.052980 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.052687 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:49.052673682 +0000 UTC m=+36.894524791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:33:48.695465 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.695236 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs"] Jul 16 20:33:48.695597 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.695531 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm"] Jul 16 20:33:48.699076 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.696306 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640a485f_3383_44c9_b7ff_1974fe02179e.slice/crio-3f38519d17816d9265b6323a61845ec066cd69c5da8fd93f615daf7afbcd4593 WatchSource:0}: Error finding container 3f38519d17816d9265b6323a61845ec066cd69c5da8fd93f615daf7afbcd4593: Status 404 returned error can't find the container with id 3f38519d17816d9265b6323a61845ec066cd69c5da8fd93f615daf7afbcd4593 Jul 16 20:33:48.700937 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.700831 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a84d24_de40_4cc1_91ac_a3ff3d003c2e.slice/crio-28b231bda30d4b6ca520179163aaabd489eefab0ef86a9825bf187313200cabf WatchSource:0}: Error finding container 28b231bda30d4b6ca520179163aaabd489eefab0ef86a9825bf187313200cabf: Status 404 returned error can't find the container with id 28b231bda30d4b6ca520179163aaabd489eefab0ef86a9825bf187313200cabf Jul 16 20:33:48.705237 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.705215 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbde4a0_208d_4fd8_abfa_d9883ad48241.slice/crio-57ed5253bd083bd25f69b47ddc229489b27cce70fa5296b87f7f343ac7e951b6 WatchSource:0}: Error finding container 57ed5253bd083bd25f69b47ddc229489b27cce70fa5296b87f7f343ac7e951b6: Status 404 returned error can't find the container with id 57ed5253bd083bd25f69b47ddc229489b27cce70fa5296b87f7f343ac7e951b6 Jul 16 20:33:48.706544 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.706524 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw"] Jul 16 20:33:48.708050 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.707977 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh"] Jul 16 20:33:48.708243 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.708174 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d410c6a_51fe_4d68_bc58_d72bbca0e752.slice/crio-9c0eea661d1dbb8d8936dabc118dceb80d6c3257364fa35557cad7a2cad8e5f0 WatchSource:0}: Error finding container 9c0eea661d1dbb8d8936dabc118dceb80d6c3257364fa35557cad7a2cad8e5f0: Status 404 returned error can't find the container with id 9c0eea661d1dbb8d8936dabc118dceb80d6c3257364fa35557cad7a2cad8e5f0 Jul 16 20:33:48.709230 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.709209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fdc7s"] Jul 16 20:33:48.709801 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.709767 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35dd9ae8_1456_44ff_8b64_5790858c897c.slice/crio-905982cbca56930437cf64117cde18c03f0b086ca34b97772d274fa818a1246a WatchSource:0}: Error finding container 905982cbca56930437cf64117cde18c03f0b086ca34b97772d274fa818a1246a: Status 404 returned error can't find the container with id 905982cbca56930437cf64117cde18c03f0b086ca34b97772d274fa818a1246a Jul 16 20:33:48.721935 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.721909 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-k7vcw"] Jul 16 20:33:48.722005 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.721945 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2z2lm"] Jul 16 20:33:48.722664 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.722639 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4a5426_e1fa_4a86_886b_8415a2614e7e.slice/crio-3ab01245b3dca40c592cab30210efbd442f39d066056acea29de591fea4a88f6 WatchSource:0}: Error finding container 3ab01245b3dca40c592cab30210efbd442f39d066056acea29de591fea4a88f6: Status 404 returned error can't find the container with id 3ab01245b3dca40c592cab30210efbd442f39d066056acea29de591fea4a88f6 Jul 16 20:33:48.724659 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:33:48.724643 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6eb2d8f_ddfe_403b_91ea_75cecba86f98.slice/crio-fd18f07631a0f2f380b9495ec3789ffb0c77e0eaf6caea4ae9b275ef239074e4 WatchSource:0}: Error finding container fd18f07631a0f2f380b9495ec3789ffb0c77e0eaf6caea4ae9b275ef239074e4: Status 404 returned error can't find the container with id fd18f07631a0f2f380b9495ec3789ffb0c77e0eaf6caea4ae9b275ef239074e4 Jul 16 20:33:48.871566 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.871537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:48.871687 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.871673 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:48.871734 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.871711 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:33:48.871778 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.871741 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:33:48.871818 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.871807 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:50.871786158 +0000 UTC m=+38.713637280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:33:48.871856 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.871819 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:33:48.871885 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.871877 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:50.871860429 +0000 UTC m=+38.713711550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:33:48.880035 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.880007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" event={"ID":"3d410c6a-51fe-4d68-bc58-d72bbca0e752","Type":"ContainerStarted","Data":"9c0eea661d1dbb8d8936dabc118dceb80d6c3257364fa35557cad7a2cad8e5f0"} Jul 16 20:33:48.881292 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.881270 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2z2lm" event={"ID":"e6eb2d8f-ddfe-403b-91ea-75cecba86f98","Type":"ContainerStarted","Data":"fd18f07631a0f2f380b9495ec3789ffb0c77e0eaf6caea4ae9b275ef239074e4"} Jul 16 20:33:48.882461 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.882443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" event={"ID":"8fbde4a0-208d-4fd8-abfa-d9883ad48241","Type":"ContainerStarted","Data":"57ed5253bd083bd25f69b47ddc229489b27cce70fa5296b87f7f343ac7e951b6"} Jul 16 20:33:48.885585 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.885541 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerStarted","Data":"a196cbf6d6e56036d7a7ec775f4fccdabbf44034c56a6956e1cdbfa8bb4c1a5b"} Jul 16 20:33:48.886817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.886788 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" event={"ID":"74a84d24-de40-4cc1-91ac-a3ff3d003c2e","Type":"ContainerStarted","Data":"28b231bda30d4b6ca520179163aaabd489eefab0ef86a9825bf187313200cabf"} Jul 16 20:33:48.888012 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.887989 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" event={"ID":"35dd9ae8-1456-44ff-8b64-5790858c897c","Type":"ContainerStarted","Data":"905982cbca56930437cf64117cde18c03f0b086ca34b97772d274fa818a1246a"} Jul 16 20:33:48.889151 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.889113 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" event={"ID":"640a485f-3383-44c9-b7ff-1974fe02179e","Type":"ContainerStarted","Data":"3f38519d17816d9265b6323a61845ec066cd69c5da8fd93f615daf7afbcd4593"} Jul 16 20:33:48.890305 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.890280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" event={"ID":"da4a5426-e1fa-4a86-886b-8415a2614e7e","Type":"ContainerStarted","Data":"3ab01245b3dca40c592cab30210efbd442f39d066056acea29de591fea4a88f6"} Jul 16 20:33:48.972865 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.972808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:48.972959 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.972900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:48.972959 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.972913 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:33:48.973071 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.972976 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:50.972957948 +0000 UTC m=+38.814809053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:33:48.973071 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.973007 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:50.97299353 +0000 UTC m=+38.814844634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:33:48.973071 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:48.973031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:48.973206 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.973110 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:48.973206 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:48.973169 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:50.973156324 +0000 UTC m=+38.815007433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:49.074382 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:49.074354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:49.074788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:49.074391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:49.074788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:49.074465 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:33:49.074788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:49.074467 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:33:49.074788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:49.074506 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:51.074494948 +0000 UTC m=+38.916346053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:33:49.074788 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:49.074517 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:51.07451216 +0000 UTC m=+38.916363265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:33:49.910821 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:49.910786 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="a196cbf6d6e56036d7a7ec775f4fccdabbf44034c56a6956e1cdbfa8bb4c1a5b" exitCode=0 Jul 16 20:33:49.911014 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:49.910851 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"a196cbf6d6e56036d7a7ec775f4fccdabbf44034c56a6956e1cdbfa8bb4c1a5b"} Jul 16 20:33:50.896594 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.896321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:50.897144 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.896725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:50.897144 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.896566 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:33:50.897144 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.897006 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:33:50.897144 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.897061 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:54.897043433 +0000 UTC m=+42.738894537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:33:50.897801 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.896988 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:33:50.897801 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.897551 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:54.897531797 +0000 UTC m=+42.739382916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:33:50.916401 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.916278 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" event={"ID":"3d410c6a-51fe-4d68-bc58-d72bbca0e752","Type":"ContainerStarted","Data":"6c2590a85aaed23120597767296763e95eee9bf4e58d285c2e736ea0c501188c"} Jul 16 20:33:50.922047 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.922017 2582 generic.go:356] "Generic (PLEG): container finished" podID="62d02010-e6f6-4108-9904-2e1e32aa50de" containerID="062881d65dbfb95d4c69048064ea1a246ed24252c93b0bf76e1ac518c8e7c5dc" exitCode=0 Jul 16 20:33:50.922163 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.922124 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerDied","Data":"062881d65dbfb95d4c69048064ea1a246ed24252c93b0bf76e1ac518c8e7c5dc"} Jul 16 20:33:50.931779 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.931711 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qcqxh" podStartSLOduration=29.235571586 podStartE2EDuration="30.931693112s" totalImagesPullingTime="1.696121526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:50.930640839 +0000 UTC m=+38.772491969" watchObservedRunningTime="2026-07-16 20:33:50.931693112 +0000 UTC m=+38.773544239" Jul 16 20:33:50.998174 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.998029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:50.998174 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.998121 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:50.998397 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:50.998264 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:50.998397 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.998338 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:50.998494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.998404 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:54.998383041 +0000 UTC m=+42.840234170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:50.998494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.998489 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:33:50.998635 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.998542 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:54.998525656 +0000 UTC m=+42.840376779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:33:50.998635 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:50.998593 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:54.998563993 +0000 UTC m=+42.840415099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:33:51.099896 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:51.099837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:51.100076 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:51.099910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:51.102088 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:51.101549 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:33:51.102088 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:51.101626 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:55.101606521 +0000 UTC m=+42.943457627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:33:51.102088 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:51.101995 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:33:51.102088 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:51.102045 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:55.10202959 +0000 UTC m=+42.943880713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:33:54.935666 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:54.935620 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:54.935681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:54.935765 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:54.935781 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:54.935784 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:54.935835 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:02.93581818 +0000 UTC m=+50.777669286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:33:54.936128 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:54.935852 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:02.935845991 +0000 UTC m=+50.777697097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:33:55.036961 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:55.036922 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:55.037160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:55.037004 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:33:55.037160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:55.037082 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:33:55.037160 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.037115 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:03.037088898 +0000 UTC m=+50.878940003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:33:55.037336 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.037176 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:33:55.037336 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.037182 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:55.037336 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.037231 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:03.037216621 +0000 UTC m=+50.879067729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:33:55.037336 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.037249 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:03.037239847 +0000 UTC m=+50.879090953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:33:55.138528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:55.138494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:33:55.138528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:55.138540 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:33:55.138790 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.138666 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:33:55.138790 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.138669 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:33:55.138790 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.138725 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:34:03.138711584 +0000 UTC m=+50.980562692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:33:55.138790 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:55.138739 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:03.138733051 +0000 UTC m=+50.980584156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:33:56.941622 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.941520 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2z2lm" event={"ID":"e6eb2d8f-ddfe-403b-91ea-75cecba86f98","Type":"ContainerStarted","Data":"ca1510401842991f2ad01feefacc8a4023ffa8aba21687cb752d72712dde0a3f"} Jul 16 20:33:56.943428 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.943392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" event={"ID":"8fbde4a0-208d-4fd8-abfa-d9883ad48241","Type":"ContainerStarted","Data":"ba2099da271f513702399c05e746c3fbd39f526a2df7007af76c6655d098069a"} Jul 16 20:33:56.947159 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.947130 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szbgs" event={"ID":"62d02010-e6f6-4108-9904-2e1e32aa50de","Type":"ContainerStarted","Data":"85f59ec32e81e4eafd67c7f286a16073357194ec43926beddecdeb179f4ae5b5"} Jul 16 20:33:56.948708 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.948686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" event={"ID":"74a84d24-de40-4cc1-91ac-a3ff3d003c2e","Type":"ContainerStarted","Data":"d1ad657e713946057ba3b6c5f09bbf1e85017deef638b7de1568ec02cb4b060a"} Jul 16 20:33:56.950312 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.950294 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/0.log" Jul 16 20:33:56.950407 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.950332 2582 generic.go:356] "Generic (PLEG): container finished" podID="35dd9ae8-1456-44ff-8b64-5790858c897c" containerID="f0d31dc38da071ca8e3035c43002b9902f1c7ec5780428583dad8af910b14666" exitCode=255 Jul 16 20:33:56.950407 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.950398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" event={"ID":"35dd9ae8-1456-44ff-8b64-5790858c897c","Type":"ContainerDied","Data":"f0d31dc38da071ca8e3035c43002b9902f1c7ec5780428583dad8af910b14666"} Jul 16 20:33:56.950550 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.950528 2582 scope.go:117] "RemoveContainer" containerID="f0d31dc38da071ca8e3035c43002b9902f1c7ec5780428583dad8af910b14666" Jul 16 20:33:56.951896 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.951873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" event={"ID":"640a485f-3383-44c9-b7ff-1974fe02179e","Type":"ContainerStarted","Data":"24a26770213fbf573d52eb5a0d04975605231fed81217435dd3db4a11cc22457"} Jul 16 20:33:56.953549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.953528 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" event={"ID":"da4a5426-e1fa-4a86-886b-8415a2614e7e","Type":"ContainerStarted","Data":"0d584f88c097bbbb9e5b128bb9c14ddef52fcec679a80bc48325ffc87f8fc82a"} Jul 16 20:33:56.960518 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:56.960472 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2z2lm" podStartSLOduration=2.376704229 podStartE2EDuration="9.960454397s" totalImagesPullingTime="7.583750168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:56.959441893 +0000 UTC m=+44.801293020" watchObservedRunningTime="2026-07-16 20:33:56.960454397 +0000 UTC m=+44.802305526" Jul 16 20:33:57.054485 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.054424 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-stvbm" podStartSLOduration=29.440479474 podStartE2EDuration="37.054402385s" totalImagesPullingTime="7.613922911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:57.008622615 +0000 UTC m=+44.850473743" watchObservedRunningTime="2026-07-16 20:33:57.054402385 +0000 UTC m=+44.896253513" Jul 16 20:33:57.083606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.082068 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-szbgs" podStartSLOduration=15.060721304 podStartE2EDuration="45.082046891s" totalImagesPullingTime="30.021325587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:57.054824257 +0000 UTC m=+44.896675386" watchObservedRunningTime="2026-07-16 20:33:57.082046891 +0000 UTC m=+44.923898019" Jul 16 20:33:57.104381 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.104313 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" podStartSLOduration=29.502944749 podStartE2EDuration="37.104291152s" totalImagesPullingTime="7.601346403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:57.083433872 +0000 UTC m=+44.925284981" watchObservedRunningTime="2026-07-16 20:33:57.104291152 +0000 UTC m=+44.946142279" Jul 16 20:33:57.128481 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.128428 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" podStartSLOduration=29.543549699 podStartE2EDuration="37.128412965s" totalImagesPullingTime="7.584863266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:57.106238819 +0000 UTC m=+44.948089947" watchObservedRunningTime="2026-07-16 20:33:57.128412965 +0000 UTC m=+44.970264070" Jul 16 20:33:57.128968 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.128928 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" podStartSLOduration=29.522317018 podStartE2EDuration="37.128916603s" totalImagesPullingTime="7.606599585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:57.127845947 +0000 UTC m=+44.969697076" watchObservedRunningTime="2026-07-16 20:33:57.128916603 +0000 UTC m=+44.970767730" Jul 16 20:33:57.363272 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.363234 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lxwv5"] Jul 16 20:33:57.381278 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.381248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.385203 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.384140 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 20:33:57.385203 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.384829 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ljszm\"" Jul 16 20:33:57.385203 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.385036 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 20:33:57.386066 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.386035 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxwv5"] Jul 16 20:33:57.460662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.460636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-data-volume\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.460794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.460665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.460794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.460688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnwlj\" (UniqueName: \"kubernetes.io/projected/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-api-access-xnwlj\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.460922 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.460805 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.460922 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.460874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-crio-socket\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.497679 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.497656 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:57.497787 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.497704 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:33:57.561794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.561768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-crio-socket\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.561939 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.561919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-data-volume\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.561998 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.561950 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.561998 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.561975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnwlj\" (UniqueName: \"kubernetes.io/projected/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-api-access-xnwlj\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.562100 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.562011 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-crio-socket\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.562100 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.562046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.562199 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:57.562104 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 20:33:57.562199 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:57.562136 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 20:33:57.562199 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:57.562163 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:58.0621412 +0000 UTC m=+45.903992308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : configmap "kube-rbac-proxy" not found Jul 16 20:33:57.562199 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:57.562180 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:58.062170589 +0000 UTC m=+45.904021701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : secret "insights-runtime-extractor-tls" not found Jul 16 20:33:57.562428 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.562298 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-data-volume\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.573354 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.573300 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnwlj\" (UniqueName: \"kubernetes.io/projected/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-api-access-xnwlj\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:57.963680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.963602 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:33:57.964112 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.964091 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/0.log" Jul 16 20:33:57.964176 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.964136 2582 generic.go:356] "Generic (PLEG): container finished" podID="35dd9ae8-1456-44ff-8b64-5790858c897c" containerID="6a3028fbe1889679caae8aba2c31cc0d7c6f94acb2e6c2ba8ef3b4e2e07b8587" exitCode=255 Jul 16 20:33:57.964253 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.964231 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" event={"ID":"35dd9ae8-1456-44ff-8b64-5790858c897c","Type":"ContainerDied","Data":"6a3028fbe1889679caae8aba2c31cc0d7c6f94acb2e6c2ba8ef3b4e2e07b8587"} Jul 16 20:33:57.964339 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.964326 2582 scope.go:117] "RemoveContainer" containerID="f0d31dc38da071ca8e3035c43002b9902f1c7ec5780428583dad8af910b14666" Jul 16 20:33:57.964529 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:57.964510 2582 scope.go:117] "RemoveContainer" containerID="6a3028fbe1889679caae8aba2c31cc0d7c6f94acb2e6c2ba8ef3b4e2e07b8587" Jul 16 20:33:57.964716 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:57.964692 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-fdc7s_openshift-console-operator(35dd9ae8-1456-44ff-8b64-5790858c897c)\"" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" podUID="35dd9ae8-1456-44ff-8b64-5790858c897c" Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:58.067277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:58.067375 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:58.067432 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:59.06741274 +0000 UTC m=+46.909263851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : configmap "kube-rbac-proxy" not found Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:58.067711 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:58.067828 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 20:33:58.067914 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:58.067873 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:59.06785925 +0000 UTC m=+46.909710362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : secret "insights-runtime-extractor-tls" not found Jul 16 20:33:58.944149 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:58.944121 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gvsbw_b9b15667-eba3-4608-a2f7-2c955b9705dc/dns-node-resolver/0.log" Jul 16 20:33:58.968763 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:58.968738 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:33:58.969086 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:58.969065 2582 scope.go:117] "RemoveContainer" containerID="6a3028fbe1889679caae8aba2c31cc0d7c6f94acb2e6c2ba8ef3b4e2e07b8587" Jul 16 20:33:58.969247 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:58.969227 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-fdc7s_openshift-console-operator(35dd9ae8-1456-44ff-8b64-5790858c897c)\"" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" podUID="35dd9ae8-1456-44ff-8b64-5790858c897c" Jul 16 20:33:59.076633 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:59.076599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:59.076784 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:59.076658 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 20:33:59.076784 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:59.076703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:33:59.076784 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:59.076716 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:01.076697936 +0000 UTC m=+48.918549064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : configmap "kube-rbac-proxy" not found Jul 16 20:33:59.076784 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:59.076781 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 20:33:59.077027 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:33:59.076812 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:01.076804317 +0000 UTC m=+48.918655422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : secret "insights-runtime-extractor-tls" not found Jul 16 20:33:59.747868 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:33:59.747841 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-npv56_5d55b6a6-cbd1-41e1-97fc-80c64efeca7b/node-ca/0.log" Jul 16 20:34:00.543994 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:00.543962 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2z2lm_e6eb2d8f-ddfe-403b-91ea-75cecba86f98/serve-healthcheck-canary/0.log" Jul 16 20:34:01.095026 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:01.094993 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:01.095190 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:01.095060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:01.095190 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:01.095114 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 20:34:01.095259 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:01.095192 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:05.095175456 +0000 UTC m=+52.937026571 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : configmap "kube-rbac-proxy" not found Jul 16 20:34:01.095259 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:01.095226 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 20:34:01.095326 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:01.095277 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:05.095263681 +0000 UTC m=+52.937114803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : secret "insights-runtime-extractor-tls" not found Jul 16 20:34:03.011982 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.011947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.011997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.012092 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.012109 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-797689db89-wq6pw: secret "image-registry-tls" not found Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.012145 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.012154 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls podName:e7820f8f-88b3-4aa1-900d-c63717610f89 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.012139618 +0000 UTC m=+66.853990724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls") pod "image-registry-797689db89-wq6pw" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89") : secret "image-registry-tls" not found Jul 16 20:34:03.012352 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.012194 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls podName:8f8c92be-67a7-4233-a49c-cd1fbfcf4547 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.012183305 +0000 UTC m=+66.854034411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fh979" (UID: "8f8c92be-67a7-4233-a49c-cd1fbfcf4547") : secret "samples-operator-tls" not found Jul 16 20:34:03.113218 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.113184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:03.113361 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.113259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:03.113361 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.113295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:34:03.113361 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.113321 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 20:34:03.113494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.113380 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 20:34:03.113494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.113383 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.113367773 +0000 UTC m=+66.955218878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : secret "router-metrics-certs-default" not found Jul 16 20:34:03.113494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.113432 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle podName:c70ee74a-e1b1-4c18-a631-cb4e6dd97804 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.11342209 +0000 UTC m=+66.955273196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle") pod "router-default-665df85b66-nl88f" (UID: "c70ee74a-e1b1-4c18-a631-cb4e6dd97804") : configmap references non-existent config key: service-ca.crt Jul 16 20:34:03.113494 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.113441 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls podName:41e2fac7-5f11-405f-aa74-f4241008bf24 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.113436347 +0000 UTC m=+66.955287451 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9jdqp" (UID: "41e2fac7-5f11-405f-aa74-f4241008bf24") : secret "cluster-monitoring-operator-tls" not found Jul 16 20:34:03.213886 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.213854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:34:03.214011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:03.213901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:34:03.214011 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.213973 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 20:34:03.214093 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.214015 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert podName:d839394e-abbe-4b8b-8a60-e52d3fe9f13d nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.214004056 +0000 UTC m=+67.055855161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-ztxn5" (UID: "d839394e-abbe-4b8b-8a60-e52d3fe9f13d") : secret "networking-console-plugin-cert" not found Jul 16 20:34:03.214093 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.213973 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 20:34:03.214093 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:03.214081 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls podName:4fc407f8-7328-46c2-bbf9-53e6bae72070 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:19.214068165 +0000 UTC m=+67.055919270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls") pod "dns-default-x958g" (UID: "4fc407f8-7328-46c2-bbf9-53e6bae72070") : secret "dns-default-metrics-tls" not found Jul 16 20:34:05.131797 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:05.131763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:05.132203 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:05.131866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:05.132203 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:05.131967 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 20:34:05.132203 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:34:05.132022 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls podName:bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1 nodeName:}" failed. No retries permitted until 2026-07-16 20:34:13.132007833 +0000 UTC m=+60.973858937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxwv5" (UID: "bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1") : secret "insights-runtime-extractor-tls" not found Jul 16 20:34:05.132317 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:05.132221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:07.497677 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:07.497644 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:34:07.498079 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:07.497703 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:34:07.498079 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:07.498014 2582 scope.go:117] "RemoveContainer" containerID="6a3028fbe1889679caae8aba2c31cc0d7c6f94acb2e6c2ba8ef3b4e2e07b8587" Jul 16 20:34:08.003816 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:08.003788 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:34:08.003966 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:08.003865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" event={"ID":"35dd9ae8-1456-44ff-8b64-5790858c897c","Type":"ContainerStarted","Data":"5699c3bd88caeacbcca4e5ab3a2a55d0c5d11515fb12009cf180e600cc535224"} Jul 16 20:34:08.004177 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:08.004156 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:34:08.009114 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:08.009092 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" Jul 16 20:34:08.050326 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:08.050247 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-fdc7s" podStartSLOduration=40.454230954 podStartE2EDuration="48.050225947s" totalImagesPullingTime="7.595994993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:08.027860245 +0000 UTC m=+55.869711372" watchObservedRunningTime="2026-07-16 20:34:08.050225947 +0000 UTC m=+55.892077075" Jul 16 20:34:11.870526 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:11.870501 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r5n26" Jul 16 20:34:13.203374 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:13.203329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:13.205530 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:13.205512 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxwv5\" (UID: \"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1\") " pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:13.296319 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:13.296292 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ljszm\"" Jul 16 20:34:13.304031 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:13.304004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxwv5" Jul 16 20:34:13.424943 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:13.424908 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxwv5"] Jul 16 20:34:13.428147 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:13.428120 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc55b10a_cf1f_47f8_9c1a_0b2c0d060ea1.slice/crio-8d70ce7ffb8ae83174356172a77d3ab80830b8cbfe40f57cfbc37218361901f0 WatchSource:0}: Error finding container 8d70ce7ffb8ae83174356172a77d3ab80830b8cbfe40f57cfbc37218361901f0: Status 404 returned error can't find the container with id 8d70ce7ffb8ae83174356172a77d3ab80830b8cbfe40f57cfbc37218361901f0 Jul 16 20:34:14.031867 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:14.031524 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxwv5" event={"ID":"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1","Type":"ContainerStarted","Data":"8d70ce7ffb8ae83174356172a77d3ab80830b8cbfe40f57cfbc37218361901f0"} Jul 16 20:34:18.349761 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.349723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:34:18.350155 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.349895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:34:18.352534 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.352517 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 20:34:18.353714 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.353698 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 20:34:18.361870 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.361852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9238df2-51ed-4a8b-870b-15cf82c175df-metrics-certs\") pod \"network-metrics-daemon-9sqrn\" (UID: \"c9238df2-51ed-4a8b-870b-15cf82c175df\") " pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:34:18.362203 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.362183 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5556d69e-b310-4298-888e-5f47489a878b-original-pull-secret\") pod \"global-pull-secret-syncer-84n8n\" (UID: \"5556d69e-b310-4298-888e-5f47489a878b\") " pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:34:18.451075 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.451021 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:34:18.453198 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.453181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5n59\" (UniqueName: \"kubernetes.io/projected/a52312b6-20e0-4f0f-847b-a9743c34320b-kube-api-access-g5n59\") pod \"network-check-target-pjcxx\" (UID: \"a52312b6-20e0-4f0f-847b-a9743c34320b\") " pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:34:18.627691 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.627658 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n8n" Jul 16 20:34:18.636819 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.636796 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ss679\"" Jul 16 20:34:18.641775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.641723 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xxdx\"" Jul 16 20:34:18.644628 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.644603 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9sqrn" Jul 16 20:34:18.650495 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.650419 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:34:18.804635 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.804550 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84n8n"] Jul 16 20:34:18.809672 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:18.809644 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5556d69e_b310_4298_888e_5f47489a878b.slice/crio-5173826bf01c130db95df3c7717f85d853e1d783eed455e2217a4f1ec566ebfa WatchSource:0}: Error finding container 5173826bf01c130db95df3c7717f85d853e1d783eed455e2217a4f1ec566ebfa: Status 404 returned error can't find the container with id 5173826bf01c130db95df3c7717f85d853e1d783eed455e2217a4f1ec566ebfa Jul 16 20:34:18.827646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.827519 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9sqrn"] Jul 16 20:34:18.832430 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:18.832397 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9238df2_51ed_4a8b_870b_15cf82c175df.slice/crio-edb61ed9935c437ae6ace73e45fb1f069bf68b04d2ffdc193971306726c72612 WatchSource:0}: Error finding container edb61ed9935c437ae6ace73e45fb1f069bf68b04d2ffdc193971306726c72612: Status 404 returned error can't find the container with id edb61ed9935c437ae6ace73e45fb1f069bf68b04d2ffdc193971306726c72612 Jul 16 20:34:18.849037 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:18.848983 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pjcxx"] Jul 16 20:34:18.853389 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:18.853365 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52312b6_20e0_4f0f_847b_a9743c34320b.slice/crio-dc3495436b2bb48aa6959ead2866440f1e2a24d7f22706b24c2b3cb19ca61137 WatchSource:0}: Error finding container dc3495436b2bb48aa6959ead2866440f1e2a24d7f22706b24c2b3cb19ca61137: Status 404 returned error can't find the container with id dc3495436b2bb48aa6959ead2866440f1e2a24d7f22706b24c2b3cb19ca61137 Jul 16 20:34:19.050933 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.050896 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84n8n" event={"ID":"5556d69e-b310-4298-888e-5f47489a878b","Type":"ContainerStarted","Data":"5173826bf01c130db95df3c7717f85d853e1d783eed455e2217a4f1ec566ebfa"} Jul 16 20:34:19.052930 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.052901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxwv5" event={"ID":"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1","Type":"ContainerStarted","Data":"12001993819cd37e0bf92264c196d056905c0af08b6b5b7a8d8afb01e247a7df"} Jul 16 20:34:19.054960 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.054918 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pjcxx" event={"ID":"a52312b6-20e0-4f0f-847b-a9743c34320b","Type":"ContainerStarted","Data":"1a31472eeae9d54f5d16064a2e5dec6500ef615f976a61137642fc4688990fd0"} Jul 16 20:34:19.055055 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.054972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pjcxx" event={"ID":"a52312b6-20e0-4f0f-847b-a9743c34320b","Type":"ContainerStarted","Data":"dc3495436b2bb48aa6959ead2866440f1e2a24d7f22706b24c2b3cb19ca61137"} Jul 16 20:34:19.055181 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.055164 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:34:19.056463 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.056439 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:19.056628 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.056492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:34:19.056977 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.056924 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sqrn" event={"ID":"c9238df2-51ed-4a8b-870b-15cf82c175df","Type":"ContainerStarted","Data":"edb61ed9935c437ae6ace73e45fb1f069bf68b04d2ffdc193971306726c72612"} Jul 16 20:34:19.059752 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.059706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8f8c92be-67a7-4233-a49c-cd1fbfcf4547-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fh979\" (UID: \"8f8c92be-67a7-4233-a49c-cd1fbfcf4547\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:34:19.059849 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.059802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"image-registry-797689db89-wq6pw\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:19.070625 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.070557 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pjcxx" podStartSLOduration=67.070540481 podStartE2EDuration="1m7.070540481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:19.069778181 +0000 UTC m=+66.911629309" watchObservedRunningTime="2026-07-16 20:34:19.070540481 +0000 UTC m=+66.912391610" Jul 16 20:34:19.157388 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.157356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:19.157521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.157450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:19.157521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.157510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:34:19.158404 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.158366 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-service-ca-bundle\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:19.160313 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.160288 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41e2fac7-5f11-405f-aa74-f4241008bf24-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9jdqp\" (UID: \"41e2fac7-5f11-405f-aa74-f4241008bf24\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:34:19.160474 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.160438 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70ee74a-e1b1-4c18-a631-cb4e6dd97804-metrics-certs\") pod \"router-default-665df85b66-nl88f\" (UID: \"c70ee74a-e1b1-4c18-a631-cb4e6dd97804\") " pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:19.218723 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.218698 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v7sr7\"" Jul 16 20:34:19.226654 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.226624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:19.258786 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.258320 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:34:19.258786 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.258377 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:34:19.261941 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.261916 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fc407f8-7328-46c2-bbf9-53e6bae72070-metrics-tls\") pod \"dns-default-x958g\" (UID: \"4fc407f8-7328-46c2-bbf9-53e6bae72070\") " pod="openshift-dns/dns-default-x958g" Jul 16 20:34:19.262546 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.262517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d839394e-abbe-4b8b-8a60-e52d3fe9f13d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-ztxn5\" (UID: \"d839394e-abbe-4b8b-8a60-e52d3fe9f13d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:34:19.276528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.276326 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dkwdv\"" Jul 16 20:34:19.283701 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.283673 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" Jul 16 20:34:19.312918 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.312816 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dv94z\"" Jul 16 20:34:19.320867 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.320841 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" Jul 16 20:34:19.325587 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.325547 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-s5gxc\"" Jul 16 20:34:19.334312 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.334067 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:19.390670 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.390459 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j5fc2\"" Jul 16 20:34:19.393333 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.393260 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:34:19.397363 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:19.397019 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7820f8f_88b3_4aa1_900d_c63717610f89.slice/crio-b5c353d95b6dadc78a1638b513b1a46f89da184e6ce5c03119d6762421562ae5 WatchSource:0}: Error finding container b5c353d95b6dadc78a1638b513b1a46f89da184e6ce5c03119d6762421562ae5: Status 404 returned error can't find the container with id b5c353d95b6dadc78a1638b513b1a46f89da184e6ce5c03119d6762421562ae5 Jul 16 20:34:19.397363 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.397063 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" Jul 16 20:34:19.429076 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.428862 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q8qkr\"" Jul 16 20:34:19.440633 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.436673 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x958g" Jul 16 20:34:19.513034 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.511821 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979"] Jul 16 20:34:19.538145 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.538073 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp"] Jul 16 20:34:19.585788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.585725 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-665df85b66-nl88f"] Jul 16 20:34:19.624306 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.623094 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5"] Jul 16 20:34:19.660062 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:19.660035 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x958g"] Jul 16 20:34:19.664494 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:19.664463 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc407f8_7328_46c2_bbf9_53e6bae72070.slice/crio-f51f80a82d7942502feb7d726d213cd4c0bb07cdfc9fce8b47d9b8869ceddf33 WatchSource:0}: Error finding container f51f80a82d7942502feb7d726d213cd4c0bb07cdfc9fce8b47d9b8869ceddf33: Status 404 returned error can't find the container with id f51f80a82d7942502feb7d726d213cd4c0bb07cdfc9fce8b47d9b8869ceddf33 Jul 16 20:34:20.063380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.063335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" event={"ID":"d839394e-abbe-4b8b-8a60-e52d3fe9f13d","Type":"ContainerStarted","Data":"4db80f408690fecf3a8ecf7cd3fbab0298337ce93b805998161a54d8f16c7bad"} Jul 16 20:34:20.065218 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.065183 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x958g" event={"ID":"4fc407f8-7328-46c2-bbf9-53e6bae72070","Type":"ContainerStarted","Data":"f51f80a82d7942502feb7d726d213cd4c0bb07cdfc9fce8b47d9b8869ceddf33"} Jul 16 20:34:20.068397 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.068371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-665df85b66-nl88f" event={"ID":"c70ee74a-e1b1-4c18-a631-cb4e6dd97804","Type":"ContainerStarted","Data":"b54fe37a402204e32245a3625348631792c33aba016b5c4bc2881e9e6d8d56aa"} Jul 16 20:34:20.068524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.068421 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-665df85b66-nl88f" event={"ID":"c70ee74a-e1b1-4c18-a631-cb4e6dd97804","Type":"ContainerStarted","Data":"796ecfa5d43e7913b2f570655564f645b7480c74484a72f3fe5d3c43eae2c6dd"} Jul 16 20:34:20.070605 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.070535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" event={"ID":"8f8c92be-67a7-4233-a49c-cd1fbfcf4547","Type":"ContainerStarted","Data":"a38e2ee3e0bb272a69711c46e4b8cfff2999c2724496a3d14cb9127bb26eb8e5"} Jul 16 20:34:20.072653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.072564 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-797689db89-wq6pw" event={"ID":"e7820f8f-88b3-4aa1-900d-c63717610f89","Type":"ContainerStarted","Data":"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7"} Jul 16 20:34:20.072653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.072624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-797689db89-wq6pw" event={"ID":"e7820f8f-88b3-4aa1-900d-c63717610f89","Type":"ContainerStarted","Data":"b5c353d95b6dadc78a1638b513b1a46f89da184e6ce5c03119d6762421562ae5"} Jul 16 20:34:20.072805 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.072705 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:20.074462 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.074361 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" event={"ID":"41e2fac7-5f11-405f-aa74-f4241008bf24","Type":"ContainerStarted","Data":"1417b2274c430825f0e39a644bc5d74b744b97093050bfd96893ed411e21f405"} Jul 16 20:34:20.088055 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.087969 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-665df85b66-nl88f" podStartSLOduration=60.087951015 podStartE2EDuration="1m0.087951015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:20.086515925 +0000 UTC m=+67.928367053" watchObservedRunningTime="2026-07-16 20:34:20.087951015 +0000 UTC m=+67.929802147" Jul 16 20:34:20.108285 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.108229 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-797689db89-wq6pw" podStartSLOduration=67.108211807 podStartE2EDuration="1m7.108211807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:20.106861903 +0000 UTC m=+67.948713032" watchObservedRunningTime="2026-07-16 20:34:20.108211807 +0000 UTC m=+67.950062935" Jul 16 20:34:20.334803 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.334678 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:20.338868 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:20.338667 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:21.082003 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:21.081746 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:21.082003 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:21.081809 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-665df85b66-nl88f" Jul 16 20:34:27.503845 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.503816 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:34:27.523958 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.523936 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:34:27.524056 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.524029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.526379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.526360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 20:34:27.526379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.526369 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 20:34:27.526554 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.526361 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v2dcd\"" Jul 16 20:34:27.527151 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.527093 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 20:34:27.527151 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.527126 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 20:34:27.527401 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.527157 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 20:34:27.527401 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.527202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 20:34:27.527401 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.527207 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 20:34:27.627642 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.627760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627655 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.627760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.627760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhnw7\" (UniqueName: \"kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.627915 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.627915 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.627787 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728631 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728724 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728648 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728724 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728711 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728807 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728807 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.728967 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.728820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhnw7\" (UniqueName: \"kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.729515 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.729461 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.729745 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.729684 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.729832 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.729802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.731527 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.731504 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.735109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.735014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.740186 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.739312 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhnw7\" (UniqueName: \"kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7\") pod \"console-67bcb66b54-c6s5z\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:27.837611 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:27.837039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:28.028253 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.028202 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:34:28.031281 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:28.031252 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bd662c_1f77_43aa_8d2d_b81a710d1a70.slice/crio-cc53cb9ab83aaeea75d51b439afce4fcb89f83fe179acbc3849ff377195766ff WatchSource:0}: Error finding container cc53cb9ab83aaeea75d51b439afce4fcb89f83fe179acbc3849ff377195766ff: Status 404 returned error can't find the container with id cc53cb9ab83aaeea75d51b439afce4fcb89f83fe179acbc3849ff377195766ff Jul 16 20:34:28.104249 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.104170 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" event={"ID":"d839394e-abbe-4b8b-8a60-e52d3fe9f13d","Type":"ContainerStarted","Data":"1b1ea5c07334a766c2a2f02d242400db73c15a1eeb866e1269989bccebb1d441"} Jul 16 20:34:28.106009 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.105982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxwv5" event={"ID":"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1","Type":"ContainerStarted","Data":"ce8c9de508ecb4229ccbb40604acc54c2de503d4ec8fc04e809fe58fd1f9ffc8"} Jul 16 20:34:28.107733 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.107703 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sqrn" event={"ID":"c9238df2-51ed-4a8b-870b-15cf82c175df","Type":"ContainerStarted","Data":"49508de613d728f3a081d00b6900c87549427447dec82e8dc99e5dd634596398"} Jul 16 20:34:28.107867 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.107752 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9sqrn" event={"ID":"c9238df2-51ed-4a8b-870b-15cf82c175df","Type":"ContainerStarted","Data":"30ae623aef5e64a528b8e69e5325608551b4194b262600adb5b1349b171f6041"} Jul 16 20:34:28.109548 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.109525 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x958g" event={"ID":"4fc407f8-7328-46c2-bbf9-53e6bae72070","Type":"ContainerStarted","Data":"b257c11a05b9547b3fea50ce160d530c445896a98bfe6ba03a94a45e33f51da5"} Jul 16 20:34:28.109686 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.109562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x958g" event={"ID":"4fc407f8-7328-46c2-bbf9-53e6bae72070","Type":"ContainerStarted","Data":"e737b5490c9ba746008e40024b403c2ee517fd48f043ddd10222ee08165ab1bd"} Jul 16 20:34:28.109686 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.109652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x958g" Jul 16 20:34:28.111331 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.111306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" event={"ID":"8f8c92be-67a7-4233-a49c-cd1fbfcf4547","Type":"ContainerStarted","Data":"0646ac7730755130e149a220ade3dc1f0fec74508909d52f9d8972f993cd0dfa"} Jul 16 20:34:28.111429 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.111352 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" event={"ID":"8f8c92be-67a7-4233-a49c-cd1fbfcf4547","Type":"ContainerStarted","Data":"2b66ecc582d6a24ee11e71ae8d939394a8beae978143add8931b7042411fdc12"} Jul 16 20:34:28.112747 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.112710 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" event={"ID":"41e2fac7-5f11-405f-aa74-f4241008bf24","Type":"ContainerStarted","Data":"f9437315c472dfc845c2c6ae1c5e3a18f8f5ffdc6bad1268e0cd827d92cfd549"} Jul 16 20:34:28.113853 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.113835 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcb66b54-c6s5z" event={"ID":"e8bd662c-1f77-43aa-8d2d-b81a710d1a70","Type":"ContainerStarted","Data":"cc53cb9ab83aaeea75d51b439afce4fcb89f83fe179acbc3849ff377195766ff"} Jul 16 20:34:28.115137 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.115118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84n8n" event={"ID":"5556d69e-b310-4298-888e-5f47489a878b","Type":"ContainerStarted","Data":"0f7081f0ff036081b132d5b959170fe070454da705cada70492bcf53ecd804ef"} Jul 16 20:34:28.120773 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.120727 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-ztxn5" podStartSLOduration=59.263076659 podStartE2EDuration="1m7.120710004s" totalImagesPullingTime="7.857633345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.120152367 +0000 UTC m=+75.962003495" watchObservedRunningTime="2026-07-16 20:34:28.120710004 +0000 UTC m=+75.962561134" Jul 16 20:34:28.138929 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.138798 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fh979" podStartSLOduration=60.185593715 podStartE2EDuration="1m8.138781362s" totalImagesPullingTime="7.953187647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.13808094 +0000 UTC m=+75.979932069" watchObservedRunningTime="2026-07-16 20:34:28.138781362 +0000 UTC m=+75.980632491" Jul 16 20:34:28.161672 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.161631 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x958g" podStartSLOduration=33.313121103 podStartE2EDuration="41.1616167s" totalImagesPullingTime="7.848495597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.159916183 +0000 UTC m=+76.001767309" watchObservedRunningTime="2026-07-16 20:34:28.1616167 +0000 UTC m=+76.003467843" Jul 16 20:34:28.180098 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.180051 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9jdqp" podStartSLOduration=60.22702763 podStartE2EDuration="1m8.1800332s" totalImagesPullingTime="7.95300557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.179757144 +0000 UTC m=+76.021608273" watchObservedRunningTime="2026-07-16 20:34:28.1800332 +0000 UTC m=+76.021884308" Jul 16 20:34:28.195788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.195742 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-84n8n" podStartSLOduration=67.471493051 podStartE2EDuration="1m16.195726381s" totalImagesPullingTime="8.72423333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.195259037 +0000 UTC m=+76.037110168" watchObservedRunningTime="2026-07-16 20:34:28.195726381 +0000 UTC m=+76.037577514" Jul 16 20:34:28.212771 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:28.212716 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9sqrn" podStartSLOduration=74.84304366 podStartE2EDuration="1m16.212694779s" totalImagesPullingTime="1.369651119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:28.211137461 +0000 UTC m=+76.052988589" watchObservedRunningTime="2026-07-16 20:34:28.212694779 +0000 UTC m=+76.054545907" Jul 16 20:34:29.097844 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.097810 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:34:29.122915 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.122082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:34:29.122915 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.122228 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.137045 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.137014 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 20:34:29.242889 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.242853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.242889 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.242903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2hd\" (UniqueName: \"kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.243187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.242972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.243187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.243020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.243187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.243066 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.243187 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.243101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.243391 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.243191 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344086 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344239 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344093 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344239 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344213 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344368 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344368 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344334 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344464 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2hd\" (UniqueName: \"kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344464 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.344935 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.344882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.345066 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.345044 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.345175 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.345140 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.345247 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.345174 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.347461 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.347438 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.347705 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.347685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.354332 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.354274 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2hd\" (UniqueName: \"kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd\") pod \"console-7bb7c49bc-dc2kd\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.442212 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.442171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:29.625656 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:29.625601 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:34:29.628662 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:29.628631 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8570ed_ccf4_4c44_97f7_776598758bd8.slice/crio-37fd3efd450e3534f2041dcb6aa9fbc0ee641e7e72620abc5681a65ca6b282d1 WatchSource:0}: Error finding container 37fd3efd450e3534f2041dcb6aa9fbc0ee641e7e72620abc5681a65ca6b282d1: Status 404 returned error can't find the container with id 37fd3efd450e3534f2041dcb6aa9fbc0ee641e7e72620abc5681a65ca6b282d1 Jul 16 20:34:30.129360 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:30.129315 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb7c49bc-dc2kd" event={"ID":"1b8570ed-ccf4-4c44-97f7-776598758bd8","Type":"ContainerStarted","Data":"37fd3efd450e3534f2041dcb6aa9fbc0ee641e7e72620abc5681a65ca6b282d1"} Jul 16 20:34:33.143567 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.143525 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxwv5" event={"ID":"bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1","Type":"ContainerStarted","Data":"3534bc7c7e0bdfe8696eae10d2b7a55aeb3cbe8be1d6a4e20008ba459ec75de1"} Jul 16 20:34:33.145120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.145094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcb66b54-c6s5z" event={"ID":"e8bd662c-1f77-43aa-8d2d-b81a710d1a70","Type":"ContainerStarted","Data":"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486"} Jul 16 20:34:33.146509 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.146488 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb7c49bc-dc2kd" event={"ID":"1b8570ed-ccf4-4c44-97f7-776598758bd8","Type":"ContainerStarted","Data":"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964"} Jul 16 20:34:33.161536 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.161493 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lxwv5" podStartSLOduration=17.399788579 podStartE2EDuration="36.161480664s" totalImagesPullingTime="18.761692085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:33.160402194 +0000 UTC m=+81.002253322" watchObservedRunningTime="2026-07-16 20:34:33.161480664 +0000 UTC m=+81.003331790" Jul 16 20:34:33.177258 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.177209 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bb7c49bc-dc2kd" podStartSLOduration=1.285656456 podStartE2EDuration="4.177193631s" totalImagesPullingTime="2.891537175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:33.176702675 +0000 UTC m=+81.018553804" watchObservedRunningTime="2026-07-16 20:34:33.177193631 +0000 UTC m=+81.019044759" Jul 16 20:34:33.195592 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:33.195536 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67bcb66b54-c6s5z" podStartSLOduration=1.705952189 podStartE2EDuration="6.195524172s" totalImagesPullingTime="4.489571983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:33.194513026 +0000 UTC m=+81.036364153" watchObservedRunningTime="2026-07-16 20:34:33.195524172 +0000 UTC m=+81.037375298" Jul 16 20:34:37.838388 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:37.838274 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:37.838388 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:37.838334 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:37.843162 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:37.843136 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:38.128057 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:38.127980 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x958g" Jul 16 20:34:38.169338 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:38.169312 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:34:39.443011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:39.442973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:39.443011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:39.443028 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:39.447860 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:39.447837 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:40.174940 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:40.174913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:34:40.227682 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:40.227654 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:34:41.085619 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:41.085595 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:34:47.488241 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:47.488205 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:34:48.721491 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.721462 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7zpk7"] Jul 16 20:34:48.850993 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.850962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.853531 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.853509 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 20:34:48.853680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.853513 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 20:34:48.854396 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.854379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 20:34:48.854709 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.854693 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-trqdw\"" Jul 16 20:34:48.854808 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.854789 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 20:34:48.896715 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-textfile\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896809 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896809 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-wtmp\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896809 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896793 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-tls\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896906 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896852 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-sys\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896906 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-metrics-client-ca\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896969 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896904 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rt4\" (UniqueName: \"kubernetes.io/projected/4508933c-f7d9-4188-ace2-b11218728255-kube-api-access-v8rt4\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.896969 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.897029 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.896993 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-root\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998099 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-textfile\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998099 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998058 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998287 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-wtmp\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998287 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998227 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-tls\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998287 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998258 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-sys\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-metrics-client-ca\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rt4\" (UniqueName: \"kubernetes.io/projected/4508933c-f7d9-4188-ace2-b11218728255-kube-api-access-v8rt4\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-wtmp\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998372 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-sys\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998385 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998436 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998347 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-textfile\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998734 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998446 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-root\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998734 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4508933c-f7d9-4188-ace2-b11218728255-root\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998874 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998858 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-metrics-client-ca\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:48.998913 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:48.998880 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:49.000856 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:49.000839 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:49.001020 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:49.000998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4508933c-f7d9-4188-ace2-b11218728255-node-exporter-tls\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:49.009910 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:49.009890 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rt4\" (UniqueName: \"kubernetes.io/projected/4508933c-f7d9-4188-ace2-b11218728255-kube-api-access-v8rt4\") pod \"node-exporter-7zpk7\" (UID: \"4508933c-f7d9-4188-ace2-b11218728255\") " pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:49.160020 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:49.159989 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7zpk7" Jul 16 20:34:49.204632 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:49.204604 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4508933c_f7d9_4188_ace2_b11218728255.slice/crio-b322ac4df4d709a129527306feeb43ea08caed443d168a033896208d86a3c6b0 WatchSource:0}: Error finding container b322ac4df4d709a129527306feeb43ea08caed443d168a033896208d86a3c6b0: Status 404 returned error can't find the container with id b322ac4df4d709a129527306feeb43ea08caed443d168a033896208d86a3c6b0 Jul 16 20:34:50.077989 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:50.077952 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pjcxx" Jul 16 20:34:50.213309 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:50.213282 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zpk7" event={"ID":"4508933c-f7d9-4188-ace2-b11218728255","Type":"ContainerStarted","Data":"b322ac4df4d709a129527306feeb43ea08caed443d168a033896208d86a3c6b0"} Jul 16 20:34:51.218199 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.218166 2582 generic.go:356] "Generic (PLEG): container finished" podID="4508933c-f7d9-4188-ace2-b11218728255" containerID="02395a489f9b4afaee1c319726891c49739e30d07cf99a94b5213034167b564a" exitCode=0 Jul 16 20:34:51.218609 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.218253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zpk7" event={"ID":"4508933c-f7d9-4188-ace2-b11218728255","Type":"ContainerDied","Data":"02395a489f9b4afaee1c319726891c49739e30d07cf99a94b5213034167b564a"} Jul 16 20:34:51.689854 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.689818 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f7c6678f8-2btww"] Jul 16 20:34:51.692600 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.692561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.694870 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.694846 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 20:34:51.694969 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.694949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 20:34:51.695050 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.694988 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 20:34:51.697182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.697166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 20:34:51.697291 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.697181 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 20:34:51.697418 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.697404 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-anav9r99m6hi8\"" Jul 16 20:34:51.697473 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.697452 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6bvlg\"" Jul 16 20:34:51.709221 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.709196 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f7c6678f8-2btww"] Jul 16 20:34:51.817507 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817622 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-grpc-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1614543f-e339-48d8-8f06-cd4beb611930-metrics-client-ca\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817734 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.817772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.817760 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqwnc\" (UniqueName: \"kubernetes.io/projected/1614543f-e339-48d8-8f06-cd4beb611930-kube-api-access-fqwnc\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.918768 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1614543f-e339-48d8-8f06-cd4beb611930-metrics-client-ca\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.918875 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.918875 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918790 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqwnc\" (UniqueName: \"kubernetes.io/projected/1614543f-e339-48d8-8f06-cd4beb611930-kube-api-access-fqwnc\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.918875 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918853 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.919039 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.919039 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.919039 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.918951 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.919193 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.919065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-grpc-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.919479 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.919454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1614543f-e339-48d8-8f06-cd4beb611930-metrics-client-ca\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.921752 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.921717 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.921858 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.921794 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.922072 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.922038 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.922171 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.922156 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.922275 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.922253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-grpc-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.922405 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.922390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1614543f-e339-48d8-8f06-cd4beb611930-secret-thanos-querier-tls\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:51.926530 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:51.926509 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqwnc\" (UniqueName: \"kubernetes.io/projected/1614543f-e339-48d8-8f06-cd4beb611930-kube-api-access-fqwnc\") pod \"thanos-querier-5f7c6678f8-2btww\" (UID: \"1614543f-e339-48d8-8f06-cd4beb611930\") " pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:52.001636 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.001535 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:34:52.129912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.129889 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f7c6678f8-2btww"] Jul 16 20:34:52.131748 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:52.131725 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1614543f_e339_48d8_8f06_cd4beb611930.slice/crio-f072b8197adb7a6ab4abf1f7296a1fbdfa3f246e484ccb55334b17e5b75c699a WatchSource:0}: Error finding container f072b8197adb7a6ab4abf1f7296a1fbdfa3f246e484ccb55334b17e5b75c699a: Status 404 returned error can't find the container with id f072b8197adb7a6ab4abf1f7296a1fbdfa3f246e484ccb55334b17e5b75c699a Jul 16 20:34:52.223971 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.223936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zpk7" event={"ID":"4508933c-f7d9-4188-ace2-b11218728255","Type":"ContainerStarted","Data":"81e413ab82dbfec96677a1feb671465179776b6110b9a5d528516802bdd7f469"} Jul 16 20:34:52.224325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.224001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zpk7" event={"ID":"4508933c-f7d9-4188-ace2-b11218728255","Type":"ContainerStarted","Data":"cfbfd6cf57d67e3d6a8b166fe6471b8605a9c2970e362cdf0a1efb1626d5017c"} Jul 16 20:34:52.225022 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.225001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"f072b8197adb7a6ab4abf1f7296a1fbdfa3f246e484ccb55334b17e5b75c699a"} Jul 16 20:34:52.241567 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:52.241526 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7zpk7" podStartSLOduration=3.220663586 podStartE2EDuration="4.241511626s" totalImagesPullingTime="956.357527ms" totalInitContainerRuntime="64.490513ms" isStatefulPod=true podCreationTimestamp="2026-07-16 20:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:52.240242978 +0000 UTC m=+100.082094109" watchObservedRunningTime="2026-07-16 20:34:52.241511626 +0000 UTC m=+100.083362752" Jul 16 20:34:53.199846 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.199815 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76bff8ddc6-dr2lk"] Jul 16 20:34:53.202193 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.202177 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.205005 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.204979 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 20:34:53.205180 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.205159 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 20:34:53.205424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.205407 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 20:34:53.206150 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.206131 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 20:34:53.206150 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.206143 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fnn56vfqnl26i\"" Jul 16 20:34:53.206329 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.206228 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-s4swz\"" Jul 16 20:34:53.226390 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.226362 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76bff8ddc6-dr2lk"] Jul 16 20:34:53.329925 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.329888 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-client-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.329961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-client-certs\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.329996 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jplgz\" (UniqueName: \"kubernetes.io/projected/90864d69-aee5-46e7-9a37-fb46c3d973f4-kube-api-access-jplgz\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330205 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.330180 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330395 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.330367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-tls\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330519 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.330410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90864d69-aee5-46e7-9a37-fb46c3d973f4-audit-log\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.330519 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.330444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-metrics-server-audit-profiles\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431152 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-client-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-client-certs\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jplgz\" (UniqueName: \"kubernetes.io/projected/90864d69-aee5-46e7-9a37-fb46c3d973f4-kube-api-access-jplgz\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431502 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-tls\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431502 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90864d69-aee5-46e7-9a37-fb46c3d973f4-audit-log\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431502 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-metrics-server-audit-profiles\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.431942 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.431907 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/90864d69-aee5-46e7-9a37-fb46c3d973f4-audit-log\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.432300 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.432246 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.433825 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.433768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/90864d69-aee5-46e7-9a37-fb46c3d973f4-metrics-server-audit-profiles\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.434320 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.434300 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-tls\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.434442 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.434425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-secret-metrics-server-client-certs\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.434506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.434484 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90864d69-aee5-46e7-9a37-fb46c3d973f4-client-ca-bundle\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.439423 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.439393 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jplgz\" (UniqueName: \"kubernetes.io/projected/90864d69-aee5-46e7-9a37-fb46c3d973f4-kube-api-access-jplgz\") pod \"metrics-server-76bff8ddc6-dr2lk\" (UID: \"90864d69-aee5-46e7-9a37-fb46c3d973f4\") " pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.464293 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.464238 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:34:53.467467 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.467447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.481782 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.481763 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:34:53.512545 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.512511 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:34:53.532438 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkj4d\" (UniqueName: \"kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532537 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532641 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532641 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532629 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532711 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532676 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.532748 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.532730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.633689 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.633834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkj4d\" (UniqueName: \"kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.633834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.633834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.633834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.634050 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633850 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.634050 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.633891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.634761 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.634704 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.634887 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.634795 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.634887 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.634831 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.635196 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.635173 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.636792 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.636770 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.637053 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.637002 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.646354 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.646209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76bff8ddc6-dr2lk"] Jul 16 20:34:53.647350 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.647326 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkj4d\" (UniqueName: \"kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d\") pod \"console-58cf74445d-cb5x9\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:53.648589 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:53.648544 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90864d69_aee5_46e7_9a37_fb46c3d973f4.slice/crio-7eb05321b58c25e87a1135647c46e6e6f48b8e83382ab85cf263d8d752ee5048 WatchSource:0}: Error finding container 7eb05321b58c25e87a1135647c46e6e6f48b8e83382ab85cf263d8d752ee5048: Status 404 returned error can't find the container with id 7eb05321b58c25e87a1135647c46e6e6f48b8e83382ab85cf263d8d752ee5048 Jul 16 20:34:53.780815 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:53.780734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:34:54.235972 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:54.235944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" event={"ID":"90864d69-aee5-46e7-9a37-fb46c3d973f4","Type":"ContainerStarted","Data":"7eb05321b58c25e87a1135647c46e6e6f48b8e83382ab85cf263d8d752ee5048"} Jul 16 20:34:54.303554 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:54.303532 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:34:54.307229 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:34:54.307205 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a886f7_b596_43bc_9bf3_1fc8cc1b9e61.slice/crio-0c28ff1c926b86bbcdcda27ec3bb40a5a75621fd9ea3a033f29128474ed33850 WatchSource:0}: Error finding container 0c28ff1c926b86bbcdcda27ec3bb40a5a75621fd9ea3a033f29128474ed33850: Status 404 returned error can't find the container with id 0c28ff1c926b86bbcdcda27ec3bb40a5a75621fd9ea3a033f29128474ed33850 Jul 16 20:34:55.242025 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.241984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"3539425d84fd9ea57f9269cc702c3f1b93992d8545f190eef6dece59d08ead2d"} Jul 16 20:34:55.242392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.242078 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"a3911c2c0861c6c1ab721185ee7dee114124abfb82d25d6582353b95f29cb654"} Jul 16 20:34:55.242392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.242118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"ef6805f888e0ca48f49f715296ed11d2b885d6a0ddfa2b56ef34e3cd09c9b3f5"} Jul 16 20:34:55.243514 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.243491 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cf74445d-cb5x9" event={"ID":"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61","Type":"ContainerStarted","Data":"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb"} Jul 16 20:34:55.243646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.243532 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cf74445d-cb5x9" event={"ID":"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61","Type":"ContainerStarted","Data":"0c28ff1c926b86bbcdcda27ec3bb40a5a75621fd9ea3a033f29128474ed33850"} Jul 16 20:34:55.270425 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:55.270381 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58cf74445d-cb5x9" podStartSLOduration=2.2703682130000002 podStartE2EDuration="2.270368213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:55.268139994 +0000 UTC m=+103.109991121" watchObservedRunningTime="2026-07-16 20:34:55.270368213 +0000 UTC m=+103.112219318" Jul 16 20:34:56.249325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.249287 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" event={"ID":"90864d69-aee5-46e7-9a37-fb46c3d973f4","Type":"ContainerStarted","Data":"a82416b49eb7775d55c4e66532ffa092227f8d49db46c63e3fdd5c3bea8a9c38"} Jul 16 20:34:56.252387 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.252359 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"1b4eebfa1270354ec76f0af32068c563b96aff23bf0fce30c19e6c6ca07c9171"} Jul 16 20:34:56.252522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.252416 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"42a3d0f85ba6fca481a71c3f26c34ed04187cd44ecfac7e2ad38332d21160305"} Jul 16 20:34:56.252522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.252456 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" event={"ID":"1614543f-e339-48d8-8f06-cd4beb611930","Type":"ContainerStarted","Data":"05744c8ee398e1b065c2ebc996d41ae4228d496905e96731c037b20c4342d21e"} Jul 16 20:34:56.268115 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.268066 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" podStartSLOduration=1.061383184 podStartE2EDuration="3.268051858s" totalImagesPullingTime="2.206668674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:56.26677194 +0000 UTC m=+104.108623068" watchObservedRunningTime="2026-07-16 20:34:56.268051858 +0000 UTC m=+104.109902985" Jul 16 20:34:56.289792 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:56.289744 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" podStartSLOduration=1.753544113 podStartE2EDuration="5.289730168s" totalImagesPullingTime="3.536186055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:56.288423971 +0000 UTC m=+104.130275098" watchObservedRunningTime="2026-07-16 20:34:56.289730168 +0000 UTC m=+104.131581294" Jul 16 20:34:57.257305 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:34:57.257275 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:35:03.267262 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:03.267228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f7c6678f8-2btww" Jul 16 20:35:03.781359 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:03.781322 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:35:03.781532 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:03.781411 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:35:03.786375 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:03.786345 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:35:04.289417 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:04.289392 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:35:04.336022 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:04.335989 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:35:05.248471 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:05.248406 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67bcb66b54-c6s5z" podUID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" containerName="console" containerID="cri-o://93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486" gracePeriod=15 Jul 16 20:35:06.001756 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.001736 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:35:06.149600 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149519 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.149600 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149549 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhnw7\" (UniqueName: \"kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.149600 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149584 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.149781 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149704 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.149825 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149796 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.149870 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149842 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config\") pod \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\" (UID: \"e8bd662c-1f77-43aa-8d2d-b81a710d1a70\") " Jul 16 20:35:06.150060 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.149995 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca" (OuterVolumeSpecName: "service-ca") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:06.150173 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.150124 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config" (OuterVolumeSpecName: "console-config") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:06.150234 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.150162 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:06.151979 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.151953 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7" (OuterVolumeSpecName: "kube-api-access-hhnw7") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "kube-api-access-hhnw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:35:06.152073 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.151985 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:06.152127 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.152060 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e8bd662c-1f77-43aa-8d2d-b81a710d1a70" (UID: "e8bd662c-1f77-43aa-8d2d-b81a710d1a70"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:06.250706 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250679 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.250706 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250702 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-oauth-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.250835 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250711 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-oauth-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.250835 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250720 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-console-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.250835 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250729 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhnw7\" (UniqueName: \"kubernetes.io/projected/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-kube-api-access-hhnw7\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.250835 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.250738 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bd662c-1f77-43aa-8d2d-b81a710d1a70-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:06.294147 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.294121 2582 generic.go:356] "Generic (PLEG): container finished" podID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" containerID="93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486" exitCode=0 Jul 16 20:35:06.294240 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.294167 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67bcb66b54-c6s5z" Jul 16 20:35:06.294240 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.294203 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcb66b54-c6s5z" event={"ID":"e8bd662c-1f77-43aa-8d2d-b81a710d1a70","Type":"ContainerDied","Data":"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486"} Jul 16 20:35:06.294320 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.294241 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67bcb66b54-c6s5z" event={"ID":"e8bd662c-1f77-43aa-8d2d-b81a710d1a70","Type":"ContainerDied","Data":"cc53cb9ab83aaeea75d51b439afce4fcb89f83fe179acbc3849ff377195766ff"} Jul 16 20:35:06.294320 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.294262 2582 scope.go:117] "RemoveContainer" containerID="93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486" Jul 16 20:35:06.303042 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.303024 2582 scope.go:117] "RemoveContainer" containerID="93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486" Jul 16 20:35:06.303286 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:35:06.303265 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486\": container with ID starting with 93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486 not found: ID does not exist" containerID="93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486" Jul 16 20:35:06.303334 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.303295 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486"} err="failed to get container status \"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486\": rpc error: code = NotFound desc = could not find container \"93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486\": container with ID starting with 93aa62e6c6df0debc4e888c4dfb8ae0549cf3a50cbc83453de37ac61a4885486 not found: ID does not exist" Jul 16 20:35:06.314651 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.314632 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:35:06.318001 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.317983 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67bcb66b54-c6s5z"] Jul 16 20:35:06.722273 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:06.722237 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" path="/var/lib/kubelet/pods/e8bd662c-1f77-43aa-8d2d-b81a710d1a70/volumes" Jul 16 20:35:12.508594 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.508539 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-797689db89-wq6pw" podUID="e7820f8f-88b3-4aa1-900d-c63717610f89" containerName="registry" containerID="cri-o://4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7" gracePeriod=30 Jul 16 20:35:12.763085 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.763032 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:35:12.895672 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895643 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895672 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895674 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895710 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895727 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895752 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895778 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895801 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.895891 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.895822 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7vzs\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs\") pod \"e7820f8f-88b3-4aa1-900d-c63717610f89\" (UID: \"e7820f8f-88b3-4aa1-900d-c63717610f89\") " Jul 16 20:35:12.896200 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.896170 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:12.896290 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.896263 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:12.898519 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.898491 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs" (OuterVolumeSpecName: "kube-api-access-d7vzs") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "kube-api-access-d7vzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:35:12.898643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.898544 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:12.898643 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.898554 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:35:12.898724 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.898641 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:12.898724 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.898665 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:35:12.903677 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.903656 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e7820f8f-88b3-4aa1-900d-c63717610f89" (UID: "e7820f8f-88b3-4aa1-900d-c63717610f89"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:35:12.997367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997338 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-trusted-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997360 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-bound-sa-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997367 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997371 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-certificates\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997380 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-registry-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997389 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-image-registry-private-configuration\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997398 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7vzs\" (UniqueName: \"kubernetes.io/projected/e7820f8f-88b3-4aa1-900d-c63717610f89-kube-api-access-d7vzs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997406 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7820f8f-88b3-4aa1-900d-c63717610f89-ca-trust-extracted\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:12.997562 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:12.997414 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7820f8f-88b3-4aa1-900d-c63717610f89-installation-pull-secrets\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:13.323314 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.323278 2582 generic.go:356] "Generic (PLEG): container finished" podID="e7820f8f-88b3-4aa1-900d-c63717610f89" containerID="4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7" exitCode=0 Jul 16 20:35:13.323475 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.323323 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-797689db89-wq6pw" Jul 16 20:35:13.323475 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.323359 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-797689db89-wq6pw" event={"ID":"e7820f8f-88b3-4aa1-900d-c63717610f89","Type":"ContainerDied","Data":"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7"} Jul 16 20:35:13.323475 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.323412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-797689db89-wq6pw" event={"ID":"e7820f8f-88b3-4aa1-900d-c63717610f89","Type":"ContainerDied","Data":"b5c353d95b6dadc78a1638b513b1a46f89da184e6ce5c03119d6762421562ae5"} Jul 16 20:35:13.323475 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.323443 2582 scope.go:117] "RemoveContainer" containerID="4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7" Jul 16 20:35:13.333028 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.332935 2582 scope.go:117] "RemoveContainer" containerID="4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7" Jul 16 20:35:13.333217 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:35:13.333188 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7\": container with ID starting with 4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7 not found: ID does not exist" containerID="4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7" Jul 16 20:35:13.333350 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.333228 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7"} err="failed to get container status \"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7\": rpc error: code = NotFound desc = could not find container \"4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7\": container with ID starting with 4737385d46c6cb4be0d2264760eb4e28363bd24c490d7b71d88725768f9655d7 not found: ID does not exist" Jul 16 20:35:13.346370 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.346347 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:35:13.349938 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.349915 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-797689db89-wq6pw"] Jul 16 20:35:13.513285 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.513252 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:35:13.513660 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:13.513308 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:35:14.721689 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:14.721653 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7820f8f-88b3-4aa1-900d-c63717610f89" path="/var/lib/kubelet/pods/e7820f8f-88b3-4aa1-900d-c63717610f89/volumes" Jul 16 20:35:17.341086 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:17.341051 2582 generic.go:356] "Generic (PLEG): container finished" podID="da4a5426-e1fa-4a86-886b-8415a2614e7e" containerID="0d584f88c097bbbb9e5b128bb9c14ddef52fcec679a80bc48325ffc87f8fc82a" exitCode=0 Jul 16 20:35:17.341510 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:17.341097 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" event={"ID":"da4a5426-e1fa-4a86-886b-8415a2614e7e","Type":"ContainerDied","Data":"0d584f88c097bbbb9e5b128bb9c14ddef52fcec679a80bc48325ffc87f8fc82a"} Jul 16 20:35:17.341510 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:17.341398 2582 scope.go:117] "RemoveContainer" containerID="0d584f88c097bbbb9e5b128bb9c14ddef52fcec679a80bc48325ffc87f8fc82a" Jul 16 20:35:18.346605 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:18.346560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-k7vcw" event={"ID":"da4a5426-e1fa-4a86-886b-8415a2614e7e","Type":"ContainerStarted","Data":"4e44410685bd55886d90ac321515c9ae7135b06a729e311b1d7b69f4390000f6"} Jul 16 20:35:22.362363 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:22.362328 2582 generic.go:356] "Generic (PLEG): container finished" podID="8fbde4a0-208d-4fd8-abfa-d9883ad48241" containerID="ba2099da271f513702399c05e746c3fbd39f526a2df7007af76c6655d098069a" exitCode=0 Jul 16 20:35:22.362723 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:22.362405 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" event={"ID":"8fbde4a0-208d-4fd8-abfa-d9883ad48241","Type":"ContainerDied","Data":"ba2099da271f513702399c05e746c3fbd39f526a2df7007af76c6655d098069a"} Jul 16 20:35:22.362768 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:22.362731 2582 scope.go:117] "RemoveContainer" containerID="ba2099da271f513702399c05e746c3fbd39f526a2df7007af76c6655d098069a" Jul 16 20:35:23.368035 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:23.368001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-x29cw" event={"ID":"8fbde4a0-208d-4fd8-abfa-d9883ad48241","Type":"ContainerStarted","Data":"6b664cb64adb88d48856e3b228eee200230c16d4aa055e3b98b047938d60ec0a"} Jul 16 20:35:28.393565 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:28.393528 2582 generic.go:356] "Generic (PLEG): container finished" podID="640a485f-3383-44c9-b7ff-1974fe02179e" containerID="24a26770213fbf573d52eb5a0d04975605231fed81217435dd3db4a11cc22457" exitCode=0 Jul 16 20:35:28.393945 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:28.393606 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" event={"ID":"640a485f-3383-44c9-b7ff-1974fe02179e","Type":"ContainerDied","Data":"24a26770213fbf573d52eb5a0d04975605231fed81217435dd3db4a11cc22457"} Jul 16 20:35:28.393990 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:28.393956 2582 scope.go:117] "RemoveContainer" containerID="24a26770213fbf573d52eb5a0d04975605231fed81217435dd3db4a11cc22457" Jul 16 20:35:29.356862 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.356821 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bb7c49bc-dc2kd" podUID="1b8570ed-ccf4-4c44-97f7-776598758bd8" containerName="console" containerID="cri-o://4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964" gracePeriod=15 Jul 16 20:35:29.399198 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.399176 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsgfs" event={"ID":"640a485f-3383-44c9-b7ff-1974fe02179e","Type":"ContainerStarted","Data":"e5804622a939e1e833c485b9a14198e549ff3a243a5b3ebf015309a33fdcd06d"} Jul 16 20:35:29.596664 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.596641 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:35:29.625205 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625135 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625205 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625176 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625261 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625289 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625515 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625431 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625515 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625480 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.625817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625791 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:29.625901 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625795 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:29.625901 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625835 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:29.626006 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.625985 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config" (OuterVolumeSpecName: "console-config") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:35:29.627430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.627409 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:29.627688 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.627661 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:35:29.726588 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726547 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b2hd\" (UniqueName: \"kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd\") pod \"1b8570ed-ccf4-4c44-97f7-776598758bd8\" (UID: \"1b8570ed-ccf4-4c44-97f7-776598758bd8\") " Jul 16 20:35:29.726761 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726748 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-oauth-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.726817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726764 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.726817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726772 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-oauth-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.726817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726781 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.726817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726789 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.726817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.726796 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b8570ed-ccf4-4c44-97f7-776598758bd8-console-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:29.728498 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.728477 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd" (OuterVolumeSpecName: "kube-api-access-6b2hd") pod "1b8570ed-ccf4-4c44-97f7-776598758bd8" (UID: "1b8570ed-ccf4-4c44-97f7-776598758bd8"). InnerVolumeSpecName "kube-api-access-6b2hd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:35:29.828052 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:29.828023 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b2hd\" (UniqueName: \"kubernetes.io/projected/1b8570ed-ccf4-4c44-97f7-776598758bd8-kube-api-access-6b2hd\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:35:30.404987 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.404955 2582 generic.go:356] "Generic (PLEG): container finished" podID="1b8570ed-ccf4-4c44-97f7-776598758bd8" containerID="4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964" exitCode=0 Jul 16 20:35:30.405380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.405000 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb7c49bc-dc2kd" event={"ID":"1b8570ed-ccf4-4c44-97f7-776598758bd8","Type":"ContainerDied","Data":"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964"} Jul 16 20:35:30.405380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.405005 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb7c49bc-dc2kd" Jul 16 20:35:30.405380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.405042 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb7c49bc-dc2kd" event={"ID":"1b8570ed-ccf4-4c44-97f7-776598758bd8","Type":"ContainerDied","Data":"37fd3efd450e3534f2041dcb6aa9fbc0ee641e7e72620abc5681a65ca6b282d1"} Jul 16 20:35:30.405380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.405060 2582 scope.go:117] "RemoveContainer" containerID="4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964" Jul 16 20:35:30.413359 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.413346 2582 scope.go:117] "RemoveContainer" containerID="4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964" Jul 16 20:35:30.413615 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:35:30.413598 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964\": container with ID starting with 4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964 not found: ID does not exist" containerID="4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964" Jul 16 20:35:30.413688 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.413621 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964"} err="failed to get container status \"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964\": rpc error: code = NotFound desc = could not find container \"4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964\": container with ID starting with 4c47845fb74ea6608bca05ffe1a998243acf8137bbaa9dafebd98fbc391a0964 not found: ID does not exist" Jul 16 20:35:30.424966 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.424939 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:35:30.429302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.429275 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bb7c49bc-dc2kd"] Jul 16 20:35:30.726168 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:30.726073 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b8570ed-ccf4-4c44-97f7-776598758bd8" path="/var/lib/kubelet/pods/1b8570ed-ccf4-4c44-97f7-776598758bd8/volumes" Jul 16 20:35:33.518842 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:33.518815 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:35:33.522492 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:35:33.522466 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76bff8ddc6-dr2lk" Jul 16 20:36:16.551710 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.551672 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552087 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7820f8f-88b3-4aa1-900d-c63717610f89" containerName="registry" Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552107 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7820f8f-88b3-4aa1-900d-c63717610f89" containerName="registry" Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552119 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" containerName="console" Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552128 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" containerName="console" Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552162 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b8570ed-ccf4-4c44-97f7-776598758bd8" containerName="console" Jul 16 20:36:16.552182 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552170 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b8570ed-ccf4-4c44-97f7-776598758bd8" containerName="console" Jul 16 20:36:16.552698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552235 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b8570ed-ccf4-4c44-97f7-776598758bd8" containerName="console" Jul 16 20:36:16.552698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552247 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8bd662c-1f77-43aa-8d2d-b81a710d1a70" containerName="console" Jul 16 20:36:16.552698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.552257 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7820f8f-88b3-4aa1-900d-c63717610f89" containerName="registry" Jul 16 20:36:16.556680 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.556656 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.567117 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.567094 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:16.687789 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.687754 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk7n5\" (UniqueName: \"kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.687928 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.687794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.687928 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.687816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.687928 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.687882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.688029 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.687943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.688029 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.688000 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.688105 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.688039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789255 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789189 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789438 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789438 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789412 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk7n5\" (UniqueName: \"kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789438 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789616 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789616 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.789616 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.789515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.790207 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.790181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.790297 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.790184 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.790297 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.790272 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.790494 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.790472 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.791778 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.791755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.791941 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.791924 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.796642 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.796626 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk7n5\" (UniqueName: \"kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5\") pod \"console-6fcd5df7f8-d9hf8\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.866684 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.866617 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:16.996323 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:16.996250 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:16.998960 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:36:16.998931 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86edc38d_ea55_42bd_93c9_97ec296075a3.slice/crio-6671759d3e4cec6d966bfde979106a91b67faa894cf9d1b64f7d88ca0236d25b WatchSource:0}: Error finding container 6671759d3e4cec6d966bfde979106a91b67faa894cf9d1b64f7d88ca0236d25b: Status 404 returned error can't find the container with id 6671759d3e4cec6d966bfde979106a91b67faa894cf9d1b64f7d88ca0236d25b Jul 16 20:36:17.587313 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:17.587274 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fcd5df7f8-d9hf8" event={"ID":"86edc38d-ea55-42bd-93c9-97ec296075a3","Type":"ContainerStarted","Data":"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff"} Jul 16 20:36:17.587671 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:17.587323 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fcd5df7f8-d9hf8" event={"ID":"86edc38d-ea55-42bd-93c9-97ec296075a3","Type":"ContainerStarted","Data":"6671759d3e4cec6d966bfde979106a91b67faa894cf9d1b64f7d88ca0236d25b"} Jul 16 20:36:17.606652 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:17.606605 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fcd5df7f8-d9hf8" podStartSLOduration=1.606586741 podStartE2EDuration="1.606586741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:36:17.604880183 +0000 UTC m=+185.446731303" watchObservedRunningTime="2026-07-16 20:36:17.606586741 +0000 UTC m=+185.448437862" Jul 16 20:36:26.218185 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.218153 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:26.248360 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.248327 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:36:26.252602 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.252561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.259941 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.259917 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:36:26.371834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.371834 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371836 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565w2\" (UniqueName: \"kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.372059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.372059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.372059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.372059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.372059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.371980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473251 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473251 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473204 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565w2\" (UniqueName: \"kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473251 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473231 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.473521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.473344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.474132 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.474107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.474234 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.474149 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.474234 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.474168 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.474378 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.474356 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.475773 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.475751 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.475866 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.475791 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.480675 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.480657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-565w2\" (UniqueName: \"kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2\") pod \"console-c8b549575-xbjps\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.563702 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.563669 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:26.683916 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.683890 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:36:26.686641 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:36:26.686606 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15225327_1798_425c_b9bc_dc6f8f35d95d.slice/crio-c3b340057588edfb597818315f7af5d546bb0123f8616e78b8fa5ea8950d62b0 WatchSource:0}: Error finding container c3b340057588edfb597818315f7af5d546bb0123f8616e78b8fa5ea8950d62b0: Status 404 returned error can't find the container with id c3b340057588edfb597818315f7af5d546bb0123f8616e78b8fa5ea8950d62b0 Jul 16 20:36:26.866937 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:26.866906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:27.630403 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:27.630362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8b549575-xbjps" event={"ID":"15225327-1798-425c-b9bc-dc6f8f35d95d","Type":"ContainerStarted","Data":"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4"} Jul 16 20:36:27.630815 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:27.630450 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8b549575-xbjps" event={"ID":"15225327-1798-425c-b9bc-dc6f8f35d95d","Type":"ContainerStarted","Data":"c3b340057588edfb597818315f7af5d546bb0123f8616e78b8fa5ea8950d62b0"} Jul 16 20:36:27.646244 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:27.646186 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c8b549575-xbjps" podStartSLOduration=1.6461712259999999 podStartE2EDuration="1.646171226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:36:27.645709122 +0000 UTC m=+195.487560248" watchObservedRunningTime="2026-07-16 20:36:27.646171226 +0000 UTC m=+195.488022351" Jul 16 20:36:36.563996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:36.563955 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:36.564435 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:36.564058 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:36.568394 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:36.568374 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:36.668478 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:36.668451 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:36:36.715761 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:36.715734 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:36:51.237962 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.237920 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fcd5df7f8-d9hf8" podUID="86edc38d-ea55-42bd-93c9-97ec296075a3" containerName="console" containerID="cri-o://0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff" gracePeriod=15 Jul 16 20:36:51.480080 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.480055 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:51.569557 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569524 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569557 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569567 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569602 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569651 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk7n5\" (UniqueName: \"kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569667 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569698 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.569877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.569720 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle\") pod \"86edc38d-ea55-42bd-93c9-97ec296075a3\" (UID: \"86edc38d-ea55-42bd-93c9-97ec296075a3\") " Jul 16 20:36:51.570119 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.570070 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:36:51.570119 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.570089 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:36:51.570267 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.570179 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:36:51.570375 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.570348 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config" (OuterVolumeSpecName: "console-config") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:36:51.571794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.571768 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:36:51.571904 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.571804 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:36:51.571904 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.571858 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5" (OuterVolumeSpecName: "kube-api-access-nk7n5") pod "86edc38d-ea55-42bd-93c9-97ec296075a3" (UID: "86edc38d-ea55-42bd-93c9-97ec296075a3"). InnerVolumeSpecName "kube-api-access-nk7n5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:36:51.670473 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670438 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670473 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670470 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk7n5\" (UniqueName: \"kubernetes.io/projected/86edc38d-ea55-42bd-93c9-97ec296075a3-kube-api-access-nk7n5\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670473 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670483 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670753 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670495 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86edc38d-ea55-42bd-93c9-97ec296075a3-console-oauth-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670753 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670506 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670753 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670520 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-oauth-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.670753 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.670531 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86edc38d-ea55-42bd-93c9-97ec296075a3-console-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:36:51.721832 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.721754 2582 generic.go:356] "Generic (PLEG): container finished" podID="86edc38d-ea55-42bd-93c9-97ec296075a3" containerID="0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff" exitCode=0 Jul 16 20:36:51.721832 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.721790 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fcd5df7f8-d9hf8" Jul 16 20:36:51.721832 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.721799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fcd5df7f8-d9hf8" event={"ID":"86edc38d-ea55-42bd-93c9-97ec296075a3","Type":"ContainerDied","Data":"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff"} Jul 16 20:36:51.721832 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.721828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fcd5df7f8-d9hf8" event={"ID":"86edc38d-ea55-42bd-93c9-97ec296075a3","Type":"ContainerDied","Data":"6671759d3e4cec6d966bfde979106a91b67faa894cf9d1b64f7d88ca0236d25b"} Jul 16 20:36:51.722072 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.721847 2582 scope.go:117] "RemoveContainer" containerID="0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff" Jul 16 20:36:51.730109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.730091 2582 scope.go:117] "RemoveContainer" containerID="0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff" Jul 16 20:36:51.730353 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:36:51.730335 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff\": container with ID starting with 0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff not found: ID does not exist" containerID="0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff" Jul 16 20:36:51.730392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.730360 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff"} err="failed to get container status \"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff\": rpc error: code = NotFound desc = could not find container \"0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff\": container with ID starting with 0ad601197e883f56ec391cc3d9c56e5eb689523416a79aa019e079ff411fb0ff not found: ID does not exist" Jul 16 20:36:51.741958 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.741933 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:51.745209 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:51.745190 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fcd5df7f8-d9hf8"] Jul 16 20:36:52.721610 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:36:52.721554 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86edc38d-ea55-42bd-93c9-97ec296075a3" path="/var/lib/kubelet/pods/86edc38d-ea55-42bd-93c9-97ec296075a3/volumes" Jul 16 20:37:01.740728 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:01.740684 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58cf74445d-cb5x9" podUID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" containerName="console" containerID="cri-o://1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb" gracePeriod=15 Jul 16 20:37:01.977507 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:01.977486 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:37:02.049593 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049543 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049741 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049613 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049741 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049663 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049741 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049694 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkj4d\" (UniqueName: \"kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049741 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049731 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049903 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049785 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.049903 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.049810 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca\") pod \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\" (UID: \"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61\") " Jul 16 20:37:02.050108 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.050078 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:37:02.050238 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.050205 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:37:02.050238 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.050225 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config" (OuterVolumeSpecName: "console-config") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:37:02.050338 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.050285 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca" (OuterVolumeSpecName: "service-ca") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:37:02.051805 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.051786 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:37:02.051864 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.051816 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:37:02.051900 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.051870 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d" (OuterVolumeSpecName: "kube-api-access-pkj4d") pod "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" (UID: "c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61"). InnerVolumeSpecName "kube-api-access-pkj4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:37:02.150769 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150734 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkj4d\" (UniqueName: \"kubernetes.io/projected/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-kube-api-access-pkj4d\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150769 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150762 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150769 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150774 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150786 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150797 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150808 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-oauth-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.150996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.150819 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61-console-oauth-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:02.764731 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.764695 2582 generic.go:356] "Generic (PLEG): container finished" podID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" containerID="1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb" exitCode=0 Jul 16 20:37:02.765164 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.764744 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58cf74445d-cb5x9" Jul 16 20:37:02.765164 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.764757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cf74445d-cb5x9" event={"ID":"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61","Type":"ContainerDied","Data":"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb"} Jul 16 20:37:02.765164 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.764834 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58cf74445d-cb5x9" event={"ID":"c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61","Type":"ContainerDied","Data":"0c28ff1c926b86bbcdcda27ec3bb40a5a75621fd9ea3a033f29128474ed33850"} Jul 16 20:37:02.765164 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.764864 2582 scope.go:117] "RemoveContainer" containerID="1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb" Jul 16 20:37:02.772683 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.772656 2582 scope.go:117] "RemoveContainer" containerID="1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb" Jul 16 20:37:02.772926 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:37:02.772909 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb\": container with ID starting with 1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb not found: ID does not exist" containerID="1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb" Jul 16 20:37:02.772974 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.772936 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb"} err="failed to get container status \"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb\": rpc error: code = NotFound desc = could not find container \"1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb\": container with ID starting with 1890a187fe81352f3b14ecf9187eea8b6bd62ce4076bafaae4af91ad511182fb not found: ID does not exist" Jul 16 20:37:02.785288 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.785258 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:37:02.788531 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:02.788512 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58cf74445d-cb5x9"] Jul 16 20:37:04.721062 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:04.721021 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" path="/var/lib/kubelet/pods/c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61/volumes" Jul 16 20:37:34.519802 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.519722 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg"] Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520042 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" containerName="console" Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520056 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" containerName="console" Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520071 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86edc38d-ea55-42bd-93c9-97ec296075a3" containerName="console" Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520077 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="86edc38d-ea55-42bd-93c9-97ec296075a3" containerName="console" Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520133 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a886f7-b596-43bc-9bf3-1fc8cc1b9e61" containerName="console" Jul 16 20:37:34.520236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.520143 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="86edc38d-ea55-42bd-93c9-97ec296075a3" containerName="console" Jul 16 20:37:34.522871 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.522853 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.525654 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.525628 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 20:37:34.525777 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.525676 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 20:37:34.525777 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.525689 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rwg28\"" Jul 16 20:37:34.532153 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.532129 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg"] Jul 16 20:37:34.689034 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.689007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.689154 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.689043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.689154 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.689081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wllz6\" (UniqueName: \"kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.789904 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.789879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wllz6\" (UniqueName: \"kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.790019 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.789932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.790019 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.789973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.790344 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.790326 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.790383 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.790346 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.798549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.798522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wllz6\" (UniqueName: \"kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.831659 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.831641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:34.956498 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:34.956471 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg"] Jul 16 20:37:34.958881 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:37:34.958853 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e48ca8_e9ff_4d57_ae88_8b40ec82880b.slice/crio-46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c WatchSource:0}: Error finding container 46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c: Status 404 returned error can't find the container with id 46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c Jul 16 20:37:35.892012 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:35.891979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerStarted","Data":"46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c"} Jul 16 20:37:40.913165 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:40.913129 2582 generic.go:356] "Generic (PLEG): container finished" podID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerID="05967f562d52efaa8cef290b1ee11a75071179bf41665eb04bac6f1fa0930068" exitCode=0 Jul 16 20:37:40.913586 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:40.913212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerDied","Data":"05967f562d52efaa8cef290b1ee11a75071179bf41665eb04bac6f1fa0930068"} Jul 16 20:37:43.926392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:43.926360 2582 generic.go:356] "Generic (PLEG): container finished" podID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerID="41e2f1164176ac7b64bfd6c6d23868fc3b9b5fb6cc154460b89540be21f9c84f" exitCode=0 Jul 16 20:37:43.926819 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:43.926444 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerDied","Data":"41e2f1164176ac7b64bfd6c6d23868fc3b9b5fb6cc154460b89540be21f9c84f"} Jul 16 20:37:49.954634 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:49.954602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerStarted","Data":"b2100a096fb21287193c3294b9bb72271b189e341283aeb43b16106d8ea87dd8"} Jul 16 20:37:49.972465 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:49.972419 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" podStartSLOduration=2.364665403 podStartE2EDuration="15.972403827s" totalImagesPullingTime="13.607738424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:37:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:37:49.971704011 +0000 UTC m=+277.813555137" watchObservedRunningTime="2026-07-16 20:37:49.972403827 +0000 UTC m=+277.814254952" Jul 16 20:37:50.960406 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:50.960367 2582 generic.go:356] "Generic (PLEG): container finished" podID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerID="b2100a096fb21287193c3294b9bb72271b189e341283aeb43b16106d8ea87dd8" exitCode=0 Jul 16 20:37:50.960965 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:50.960412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerDied","Data":"b2100a096fb21287193c3294b9bb72271b189e341283aeb43b16106d8ea87dd8"} Jul 16 20:37:52.086095 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.086072 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:52.132687 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.132655 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wllz6\" (UniqueName: \"kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6\") pod \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " Jul 16 20:37:52.132856 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.132766 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle\") pod \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " Jul 16 20:37:52.132856 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.132792 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util\") pod \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\" (UID: \"74e48ca8-e9ff-4d57-ae88-8b40ec82880b\") " Jul 16 20:37:52.133414 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.133392 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle" (OuterVolumeSpecName: "bundle") pod "74e48ca8-e9ff-4d57-ae88-8b40ec82880b" (UID: "74e48ca8-e9ff-4d57-ae88-8b40ec82880b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:37:52.134788 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.134764 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6" (OuterVolumeSpecName: "kube-api-access-wllz6") pod "74e48ca8-e9ff-4d57-ae88-8b40ec82880b" (UID: "74e48ca8-e9ff-4d57-ae88-8b40ec82880b"). InnerVolumeSpecName "kube-api-access-wllz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:37:52.137031 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.137011 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util" (OuterVolumeSpecName: "util") pod "74e48ca8-e9ff-4d57-ae88-8b40ec82880b" (UID: "74e48ca8-e9ff-4d57-ae88-8b40ec82880b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:37:52.233668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.233614 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:52.233668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.233633 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:52.233668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.233642 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wllz6\" (UniqueName: \"kubernetes.io/projected/74e48ca8-e9ff-4d57-ae88-8b40ec82880b-kube-api-access-wllz6\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:37:52.970214 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.970181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" event={"ID":"74e48ca8-e9ff-4d57-ae88-8b40ec82880b","Type":"ContainerDied","Data":"46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c"} Jul 16 20:37:52.970342 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.970226 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46ee513b2687f2a9cbfc7ff53707b49aedacb6d57f8d25d5b427862638fd048c" Jul 16 20:37:52.970342 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:52.970191 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnqgg" Jul 16 20:37:57.523031 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523002 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj"] Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523410 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="util" Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523427 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="util" Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523441 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="pull" Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523449 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="pull" Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523465 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="extract" Jul 16 20:37:57.523469 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523473 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="extract" Jul 16 20:37:57.523790 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.523552 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="74e48ca8-e9ff-4d57-ae88-8b40ec82880b" containerName="extract" Jul 16 20:37:57.564827 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.564804 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj"] Jul 16 20:37:57.564961 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.564902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.567460 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.567442 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 20:37:57.567915 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.567887 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:37:57.568161 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.568146 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-22bjv\"" Jul 16 20:37:57.669999 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.669971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pngf\" (UniqueName: \"kubernetes.io/projected/ea52319e-67a6-40d8-966b-28a7037a9392-kube-api-access-2pngf\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.670085 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.670028 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea52319e-67a6-40d8-966b-28a7037a9392-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.771021 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.770997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea52319e-67a6-40d8-966b-28a7037a9392-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.771122 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.771039 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pngf\" (UniqueName: \"kubernetes.io/projected/ea52319e-67a6-40d8-966b-28a7037a9392-kube-api-access-2pngf\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.771366 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.771348 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea52319e-67a6-40d8-966b-28a7037a9392-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.780618 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.780552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pngf\" (UniqueName: \"kubernetes.io/projected/ea52319e-67a6-40d8-966b-28a7037a9392-kube-api-access-2pngf\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtchj\" (UID: \"ea52319e-67a6-40d8-966b-28a7037a9392\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:57.873883 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:57.873862 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" Jul 16 20:37:58.031653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:58.031567 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj"] Jul 16 20:37:58.035534 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:37:58.035498 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea52319e_67a6_40d8_966b_28a7037a9392.slice/crio-a5e823793c71a984fd051873b802e07bdae5e01b7147bf0d6a7b741d6d36ea7c WatchSource:0}: Error finding container a5e823793c71a984fd051873b802e07bdae5e01b7147bf0d6a7b741d6d36ea7c: Status 404 returned error can't find the container with id a5e823793c71a984fd051873b802e07bdae5e01b7147bf0d6a7b741d6d36ea7c Jul 16 20:37:58.996993 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:37:58.996958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" event={"ID":"ea52319e-67a6-40d8-966b-28a7037a9392","Type":"ContainerStarted","Data":"a5e823793c71a984fd051873b802e07bdae5e01b7147bf0d6a7b741d6d36ea7c"} Jul 16 20:38:00.003067 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:00.003034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" event={"ID":"ea52319e-67a6-40d8-966b-28a7037a9392","Type":"ContainerStarted","Data":"e7e87a42b4e34398e4afd5ac37575d830145ba0dd0ae9aeaa19b2da5a36482ec"} Jul 16 20:38:00.041698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:00.041634 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtchj" podStartSLOduration=1.742197242 podStartE2EDuration="3.041615753s" totalImagesPullingTime="1.299418511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:00.040416656 +0000 UTC m=+287.882267782" watchObservedRunningTime="2026-07-16 20:38:00.041615753 +0000 UTC m=+287.883466879" Jul 16 20:38:01.160719 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.160688 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k"] Jul 16 20:38:01.163325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.163309 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.165935 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.165908 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 20:38:01.165935 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.165924 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rwg28\"" Jul 16 20:38:01.166826 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.166810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 20:38:01.173399 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.173377 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k"] Jul 16 20:38:01.198737 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.198711 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.198822 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.198741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.198822 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.198777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzj45\" (UniqueName: \"kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.271698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.271675 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f2qrw"] Jul 16 20:38:01.273865 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.273851 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.276185 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.276168 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 20:38:01.276293 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.276274 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dhz4p\"" Jul 16 20:38:01.276616 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.276598 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 20:38:01.285493 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.285472 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f2qrw"] Jul 16 20:38:01.299360 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.299341 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.299446 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.299370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.299446 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.299405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzj45\" (UniqueName: \"kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.299707 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.299692 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.299751 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.299709 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.317772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.317755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzj45\" (UniqueName: \"kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.399985 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.399963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.400068 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.400001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmtm6\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-kube-api-access-gmtm6\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.472207 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.472141 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:01.500764 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.500741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.500863 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.500784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmtm6\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-kube-api-access-gmtm6\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.510350 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.510330 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.510496 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.510477 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmtm6\" (UniqueName: \"kubernetes.io/projected/05886015-a288-436e-83a7-c9a973f64d92-kube-api-access-gmtm6\") pod \"cert-manager-webhook-6486ffd997-f2qrw\" (UID: \"05886015-a288-436e-83a7-c9a973f64d92\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.597195 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.597167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:01.603419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.603401 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k"] Jul 16 20:38:01.608087 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:38:01.608054 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb927adfe_243e_462c_9946_2284602a5814.slice/crio-38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995 WatchSource:0}: Error finding container 38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995: Status 404 returned error can't find the container with id 38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995 Jul 16 20:38:01.735542 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:01.735519 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f2qrw"] Jul 16 20:38:01.759347 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:38:01.759323 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05886015_a288_436e_83a7_c9a973f64d92.slice/crio-6533ac68407d615507f6e6a5791d0db2086cbb48365a4b6fb282203b9ff70729 WatchSource:0}: Error finding container 6533ac68407d615507f6e6a5791d0db2086cbb48365a4b6fb282203b9ff70729: Status 404 returned error can't find the container with id 6533ac68407d615507f6e6a5791d0db2086cbb48365a4b6fb282203b9ff70729 Jul 16 20:38:02.011424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:02.011355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" event={"ID":"05886015-a288-436e-83a7-c9a973f64d92","Type":"ContainerStarted","Data":"6533ac68407d615507f6e6a5791d0db2086cbb48365a4b6fb282203b9ff70729"} Jul 16 20:38:02.012690 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:02.012670 2582 generic.go:356] "Generic (PLEG): container finished" podID="b927adfe-243e-462c-9946-2284602a5814" containerID="845ed8f5d890127bb995b30c8e0c4142204edb24fd801d1897f0bbd9caedc7fa" exitCode=0 Jul 16 20:38:02.012760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:02.012707 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" event={"ID":"b927adfe-243e-462c-9946-2284602a5814","Type":"ContainerDied","Data":"845ed8f5d890127bb995b30c8e0c4142204edb24fd801d1897f0bbd9caedc7fa"} Jul 16 20:38:02.012760 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:02.012733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" event={"ID":"b927adfe-243e-462c-9946-2284602a5814","Type":"ContainerStarted","Data":"38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995"} Jul 16 20:38:03.618424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.618381 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf"] Jul 16 20:38:03.622122 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.622093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.624670 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.624644 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-djwqz\"" Jul 16 20:38:03.640785 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.640757 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf"] Jul 16 20:38:03.719213 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.719182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpvc\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-kube-api-access-mbpvc\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.719378 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.719271 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.820019 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.819989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.820168 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.820036 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpvc\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-kube-api-access-mbpvc\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.828050 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.828016 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.828173 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.828078 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpvc\" (UniqueName: \"kubernetes.io/projected/f8b76a42-c805-487b-8859-8a63319ed252-kube-api-access-mbpvc\") pod \"cert-manager-cainjector-559cdd8c87-dj4kf\" (UID: \"f8b76a42-c805-487b-8859-8a63319ed252\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:03.933219 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:03.933134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" Jul 16 20:38:04.874248 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:04.874224 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf"] Jul 16 20:38:04.877492 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:38:04.877462 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8b76a42_c805_487b_8859_8a63319ed252.slice/crio-60bcd4900e69e4d03d85d773a70fa0522769b6953db421e6b3aa66785e4b4d79 WatchSource:0}: Error finding container 60bcd4900e69e4d03d85d773a70fa0522769b6953db421e6b3aa66785e4b4d79: Status 404 returned error can't find the container with id 60bcd4900e69e4d03d85d773a70fa0522769b6953db421e6b3aa66785e4b4d79 Jul 16 20:38:05.029781 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.029696 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" event={"ID":"f8b76a42-c805-487b-8859-8a63319ed252","Type":"ContainerStarted","Data":"ce18adb93ab1573b116b7d9b41a4292d74b0c3d46613290338058f3bd26a5407"} Jul 16 20:38:05.029781 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.029747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" event={"ID":"f8b76a42-c805-487b-8859-8a63319ed252","Type":"ContainerStarted","Data":"60bcd4900e69e4d03d85d773a70fa0522769b6953db421e6b3aa66785e4b4d79"} Jul 16 20:38:05.031362 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.031334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" event={"ID":"05886015-a288-436e-83a7-c9a973f64d92","Type":"ContainerStarted","Data":"210e0604d9719ed9b102d7cb1d5d8656e58c14ac928a278194df4d85f3c15352"} Jul 16 20:38:05.031506 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.031387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:05.032988 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.032965 2582 generic.go:356] "Generic (PLEG): container finished" podID="b927adfe-243e-462c-9946-2284602a5814" containerID="d568e5e1f69b5b6702267e7580ff7c7a7592afc7fe9625c94c1d5d78fb76c30f" exitCode=0 Jul 16 20:38:05.033082 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.033047 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" event={"ID":"b927adfe-243e-462c-9946-2284602a5814","Type":"ContainerDied","Data":"d568e5e1f69b5b6702267e7580ff7c7a7592afc7fe9625c94c1d5d78fb76c30f"} Jul 16 20:38:05.046588 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.046529 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dj4kf" podStartSLOduration=2.046513934 podStartE2EDuration="2.046513934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:05.045777404 +0000 UTC m=+292.887628532" watchObservedRunningTime="2026-07-16 20:38:05.046513934 +0000 UTC m=+292.888365062" Jul 16 20:38:05.064288 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:05.064250 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" podStartSLOduration=1.079860496 podStartE2EDuration="4.064235878s" totalImagesPullingTime="2.984375382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:05.063476598 +0000 UTC m=+292.905327718" watchObservedRunningTime="2026-07-16 20:38:05.064235878 +0000 UTC m=+292.906087005" Jul 16 20:38:06.038828 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:06.038795 2582 generic.go:356] "Generic (PLEG): container finished" podID="b927adfe-243e-462c-9946-2284602a5814" containerID="3d71f36c9db3117aff91fd0f2260adef78ed5abfce6ac81c671a7098097943ed" exitCode=0 Jul 16 20:38:06.039180 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:06.038875 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" event={"ID":"b927adfe-243e-462c-9946-2284602a5814","Type":"ContainerDied","Data":"3d71f36c9db3117aff91fd0f2260adef78ed5abfce6ac81c671a7098097943ed"} Jul 16 20:38:07.159896 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.159870 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:07.246491 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.246465 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzj45\" (UniqueName: \"kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45\") pod \"b927adfe-243e-462c-9946-2284602a5814\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " Jul 16 20:38:07.246628 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.246533 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util\") pod \"b927adfe-243e-462c-9946-2284602a5814\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " Jul 16 20:38:07.246628 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.246592 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle\") pod \"b927adfe-243e-462c-9946-2284602a5814\" (UID: \"b927adfe-243e-462c-9946-2284602a5814\") " Jul 16 20:38:07.246979 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.246953 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle" (OuterVolumeSpecName: "bundle") pod "b927adfe-243e-462c-9946-2284602a5814" (UID: "b927adfe-243e-462c-9946-2284602a5814"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:38:07.248429 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.248408 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45" (OuterVolumeSpecName: "kube-api-access-wzj45") pod "b927adfe-243e-462c-9946-2284602a5814" (UID: "b927adfe-243e-462c-9946-2284602a5814"). InnerVolumeSpecName "kube-api-access-wzj45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:38:07.250804 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.250768 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util" (OuterVolumeSpecName: "util") pod "b927adfe-243e-462c-9946-2284602a5814" (UID: "b927adfe-243e-462c-9946-2284602a5814"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:38:07.348122 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.348053 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:07.348122 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.348082 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzj45\" (UniqueName: \"kubernetes.io/projected/b927adfe-243e-462c-9946-2284602a5814-kube-api-access-wzj45\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:07.348122 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:07.348092 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b927adfe-243e-462c-9946-2284602a5814-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:08.048456 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:08.048429 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" Jul 16 20:38:08.048668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:08.048424 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5656k" event={"ID":"b927adfe-243e-462c-9946-2284602a5814","Type":"ContainerDied","Data":"38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995"} Jul 16 20:38:08.048668 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:08.048545 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38f66793d996f232b3988cca7f63938aac610651244a57d2ee30e02182384995" Jul 16 20:38:11.041466 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:11.041430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f2qrw" Jul 16 20:38:12.599893 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:12.599853 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:38:12.602026 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:12.602006 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:38:12.604560 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:12.604537 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:38:12.606335 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:12.606316 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:38:12.611875 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:12.611859 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 20:38:21.176479 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176444 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8"] Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176744 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="extract" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176755 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="extract" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176776 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="util" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176781 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="util" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176787 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="pull" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176792 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="pull" Jul 16 20:38:21.178208 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.176842 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b927adfe-243e-462c-9946-2284602a5814" containerName="extract" Jul 16 20:38:21.179037 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.179019 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.181659 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.181639 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rwg28\"" Jul 16 20:38:21.181759 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.181681 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 20:38:21.182410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.182393 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 20:38:21.187781 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.187757 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8"] Jul 16 20:38:21.251608 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.251564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h545s\" (UniqueName: \"kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.251700 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.251666 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.251700 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.251684 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.352810 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.352787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h545s\" (UniqueName: \"kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.352912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.352859 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.352912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.352877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.353315 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.353295 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.353353 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.353310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.362984 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.362965 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h545s\" (UniqueName: \"kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.488756 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.488707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:21.641128 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:21.640119 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8"] Jul 16 20:38:22.108547 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:22.108520 2582 generic.go:356] "Generic (PLEG): container finished" podID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerID="dbc45d38ebee28995518e5f339b7977ea75a2b8044b7c81e520b17cc23e94892" exitCode=0 Jul 16 20:38:22.108718 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:22.108565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" event={"ID":"9e281ba1-a3da-4878-9ee2-a2938ea94983","Type":"ContainerDied","Data":"dbc45d38ebee28995518e5f339b7977ea75a2b8044b7c81e520b17cc23e94892"} Jul 16 20:38:22.108718 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:22.108621 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" event={"ID":"9e281ba1-a3da-4878-9ee2-a2938ea94983","Type":"ContainerStarted","Data":"f9c346cb5b7bfe83c882f4fb355bf9065b4f7f292458dad1c4c168ceaf918384"} Jul 16 20:38:22.109008 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:22.108991 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 20:38:23.115468 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:23.115440 2582 generic.go:356] "Generic (PLEG): container finished" podID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerID="6140411fd3c3da4ba7ffa59f0c5c30d5e12814f24ee4104afb462bcc5528105d" exitCode=0 Jul 16 20:38:23.115830 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:23.115516 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" event={"ID":"9e281ba1-a3da-4878-9ee2-a2938ea94983","Type":"ContainerDied","Data":"6140411fd3c3da4ba7ffa59f0c5c30d5e12814f24ee4104afb462bcc5528105d"} Jul 16 20:38:24.128441 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:24.128405 2582 generic.go:356] "Generic (PLEG): container finished" podID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerID="f576758a8bfed2ed24b10db204b741292de7dc4928168462ee2ba425ab805c3a" exitCode=0 Jul 16 20:38:24.128812 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:24.128484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" event={"ID":"9e281ba1-a3da-4878-9ee2-a2938ea94983","Type":"ContainerDied","Data":"f576758a8bfed2ed24b10db204b741292de7dc4928168462ee2ba425ab805c3a"} Jul 16 20:38:25.252956 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.252927 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:25.388017 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.387937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util\") pod \"9e281ba1-a3da-4878-9ee2-a2938ea94983\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " Jul 16 20:38:25.388017 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.388014 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle\") pod \"9e281ba1-a3da-4878-9ee2-a2938ea94983\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " Jul 16 20:38:25.388254 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.388048 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h545s\" (UniqueName: \"kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s\") pod \"9e281ba1-a3da-4878-9ee2-a2938ea94983\" (UID: \"9e281ba1-a3da-4878-9ee2-a2938ea94983\") " Jul 16 20:38:25.388779 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.388756 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle" (OuterVolumeSpecName: "bundle") pod "9e281ba1-a3da-4878-9ee2-a2938ea94983" (UID: "9e281ba1-a3da-4878-9ee2-a2938ea94983"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:38:25.390080 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.390055 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s" (OuterVolumeSpecName: "kube-api-access-h545s") pod "9e281ba1-a3da-4878-9ee2-a2938ea94983" (UID: "9e281ba1-a3da-4878-9ee2-a2938ea94983"). InnerVolumeSpecName "kube-api-access-h545s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:38:25.393770 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.393744 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util" (OuterVolumeSpecName: "util") pod "9e281ba1-a3da-4878-9ee2-a2938ea94983" (UID: "9e281ba1-a3da-4878-9ee2-a2938ea94983"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:38:25.489420 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.489396 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:25.489420 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.489417 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h545s\" (UniqueName: \"kubernetes.io/projected/9e281ba1-a3da-4878-9ee2-a2938ea94983-kube-api-access-h545s\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:25.489544 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:25.489426 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e281ba1-a3da-4878-9ee2-a2938ea94983-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:38:26.138440 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:26.138412 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" Jul 16 20:38:26.138440 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:26.138417 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbcfs8" event={"ID":"9e281ba1-a3da-4878-9ee2-a2938ea94983","Type":"ContainerDied","Data":"f9c346cb5b7bfe83c882f4fb355bf9065b4f7f292458dad1c4c168ceaf918384"} Jul 16 20:38:26.138654 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:26.138459 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9c346cb5b7bfe83c882f4fb355bf9065b4f7f292458dad1c4c168ceaf918384" Jul 16 20:38:34.325171 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325140 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs"] Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325431 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="util" Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325442 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="util" Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325454 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="pull" Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325459 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="pull" Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325474 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="extract" Jul 16 20:38:34.325524 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325479 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="extract" Jul 16 20:38:34.325721 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.325533 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e281ba1-a3da-4878-9ee2-a2938ea94983" containerName="extract" Jul 16 20:38:34.328381 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.328363 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.331526 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.331494 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 20:38:34.331693 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.331645 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 20:38:34.331693 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.331671 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 20:38:34.331693 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.331688 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vwh9v\"" Jul 16 20:38:34.331863 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.331749 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 20:38:34.343403 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.343384 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs"] Jul 16 20:38:34.453750 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.453725 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnn6v\" (UniqueName: \"kubernetes.io/projected/f0335755-dcf1-4f86-b7ba-f39eb260d493-kube-api-access-hnn6v\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.453859 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.453758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.453900 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.453854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.453900 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.453883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-tmp\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.453969 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.453919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-manifests\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.554846 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.554819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-manifests\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.554944 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.554858 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnn6v\" (UniqueName: \"kubernetes.io/projected/f0335755-dcf1-4f86-b7ba-f39eb260d493-kube-api-access-hnn6v\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.554944 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.554879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.554944 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.554911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.555100 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.555081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-tmp\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.555223 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.555201 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-manifests\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.557173 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.557147 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0335755-dcf1-4f86-b7ba-f39eb260d493-tmp\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.557282 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.557221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.557392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.557373 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0335755-dcf1-4f86-b7ba-f39eb260d493-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.563398 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.563377 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnn6v\" (UniqueName: \"kubernetes.io/projected/f0335755-dcf1-4f86-b7ba-f39eb260d493-kube-api-access-hnn6v\") pod \"opendatahub-operator-controller-manager-6d6477549b-rjlrs\" (UID: \"f0335755-dcf1-4f86-b7ba-f39eb260d493\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.638456 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.638410 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:34.816124 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:34.816089 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs"] Jul 16 20:38:34.818616 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:38:34.818590 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0335755_dcf1_4f86_b7ba_f39eb260d493.slice/crio-edd21c099643601349434ba889f49b41db4d68c08ffe21d6a59cdf18f135007c WatchSource:0}: Error finding container edd21c099643601349434ba889f49b41db4d68c08ffe21d6a59cdf18f135007c: Status 404 returned error can't find the container with id edd21c099643601349434ba889f49b41db4d68c08ffe21d6a59cdf18f135007c Jul 16 20:38:35.182699 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:35.182668 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" event={"ID":"f0335755-dcf1-4f86-b7ba-f39eb260d493","Type":"ContainerStarted","Data":"edd21c099643601349434ba889f49b41db4d68c08ffe21d6a59cdf18f135007c"} Jul 16 20:38:39.207107 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:39.207067 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" event={"ID":"f0335755-dcf1-4f86-b7ba-f39eb260d493","Type":"ContainerStarted","Data":"86e079dfc9e7bf1d871be7511a9b3a0dc275e9ae14fd6c6768bfb4eabb29c5ff"} Jul 16 20:38:40.213322 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:40.213286 2582 generic.go:356] "Generic (PLEG): container finished" podID="f0335755-dcf1-4f86-b7ba-f39eb260d493" containerID="86e079dfc9e7bf1d871be7511a9b3a0dc275e9ae14fd6c6768bfb4eabb29c5ff" exitCode=0 Jul 16 20:38:40.213808 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:40.213330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" event={"ID":"f0335755-dcf1-4f86-b7ba-f39eb260d493","Type":"ContainerDied","Data":"86e079dfc9e7bf1d871be7511a9b3a0dc275e9ae14fd6c6768bfb4eabb29c5ff"} Jul 16 20:38:41.219656 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:41.219624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" event={"ID":"f0335755-dcf1-4f86-b7ba-f39eb260d493","Type":"ContainerStarted","Data":"a1dd25b5db0a27375f9be65bdbc8ccd4d356bf7d3466a65d9b0962727398e131"} Jul 16 20:38:41.219985 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:41.219719 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:41.241389 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:41.241330 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" podStartSLOduration=2.178519841 podStartE2EDuration="7.241315757s" totalImagesPullingTime="3.980726179s" totalInitContainerRuntime="1.082069737s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:41.240231978 +0000 UTC m=+329.082083171" watchObservedRunningTime="2026-07-16 20:38:41.241315757 +0000 UTC m=+329.083166883" Jul 16 20:38:52.227357 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:52.227328 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6477549b-rjlrs" Jul 16 20:38:54.311084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.311055 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt"] Jul 16 20:38:54.319304 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.319281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.321770 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.321748 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 20:38:54.321871 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.321849 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 20:38:54.322607 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.322566 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 20:38:54.322765 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.322745 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-97rsl\"" Jul 16 20:38:54.322842 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.322750 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 20:38:54.322842 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.322751 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 20:38:54.329376 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.329357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt"] Jul 16 20:38:54.396662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.396639 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhncn\" (UniqueName: \"kubernetes.io/projected/08398a93-dbe9-4ed0-8400-11b601c55790-kube-api-access-nhncn\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.396762 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.396671 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-metrics-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.396762 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.396695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/08398a93-dbe9-4ed0-8400-11b601c55790-manager-config\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.396827 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.396784 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.497785 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.497743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhncn\" (UniqueName: \"kubernetes.io/projected/08398a93-dbe9-4ed0-8400-11b601c55790-kube-api-access-nhncn\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.497954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.497792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-metrics-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.497954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.497892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/08398a93-dbe9-4ed0-8400-11b601c55790-manager-config\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.497954 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.497933 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.498661 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.498638 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/08398a93-dbe9-4ed0-8400-11b601c55790-manager-config\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.500481 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.500454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-metrics-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.500567 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.500535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08398a93-dbe9-4ed0-8400-11b601c55790-cert\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.529701 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.529681 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhncn\" (UniqueName: \"kubernetes.io/projected/08398a93-dbe9-4ed0-8400-11b601c55790-kube-api-access-nhncn\") pod \"lws-controller-manager-649bd55f77-wpxkt\" (UID: \"08398a93-dbe9-4ed0-8400-11b601c55790\") " pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.628707 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.628653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:54.772610 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:54.772480 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt"] Jul 16 20:38:54.775078 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:38:54.775054 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08398a93_dbe9_4ed0_8400_11b601c55790.slice/crio-e2de9250fa3a5cee01a99e1eef8669527b89e7bb133f3a78bb166b758c80e292 WatchSource:0}: Error finding container e2de9250fa3a5cee01a99e1eef8669527b89e7bb133f3a78bb166b758c80e292: Status 404 returned error can't find the container with id e2de9250fa3a5cee01a99e1eef8669527b89e7bb133f3a78bb166b758c80e292 Jul 16 20:38:55.283265 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:55.283231 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" event={"ID":"08398a93-dbe9-4ed0-8400-11b601c55790","Type":"ContainerStarted","Data":"e2de9250fa3a5cee01a99e1eef8669527b89e7bb133f3a78bb166b758c80e292"} Jul 16 20:38:57.295939 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:57.295905 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" event={"ID":"08398a93-dbe9-4ed0-8400-11b601c55790","Type":"ContainerStarted","Data":"6718ac18dcb3d746df239b3f80b8c85dcae018a49a00e2f2f6d6c4753b73e9cf"} Jul 16 20:38:57.296378 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:57.295973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:38:57.319666 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:38:57.319624 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" podStartSLOduration=1.863635912 podStartE2EDuration="3.319607699s" totalImagesPullingTime="1.455971787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:57.318189011 +0000 UTC m=+345.160040138" watchObservedRunningTime="2026-07-16 20:38:57.319607699 +0000 UTC m=+345.161458906" Jul 16 20:39:04.515768 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.515693 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf"] Jul 16 20:39:04.518791 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.518773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.522937 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.522918 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 20:39:04.523045 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.522923 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 20:39:04.523045 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.522949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 20:39:04.523045 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.522975 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-42cc9\"" Jul 16 20:39:04.523353 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.523338 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 20:39:04.544528 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.544506 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf"] Jul 16 20:39:04.572319 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d14b7f24-e9ce-4b65-a89e-53843aadef8d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572430 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572423 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmzb7\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-kube-api-access-vmzb7\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.572662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.572546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673420 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmzb7\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-kube-api-access-vmzb7\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d14b7f24-e9ce-4b65-a89e-53843aadef8d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673549 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.673838 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.673616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.674051 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.674019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.675886 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.675863 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d14b7f24-e9ce-4b65-a89e-53843aadef8d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.675986 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.675868 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.675986 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.675962 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.676090 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.676002 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.687413 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.687391 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.689762 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.689742 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmzb7\" (UniqueName: \"kubernetes.io/projected/d14b7f24-e9ce-4b65-a89e-53843aadef8d-kube-api-access-vmzb7\") pod \"istiod-openshift-gateway-94bb8fbfd-7vrjf\" (UID: \"d14b7f24-e9ce-4b65-a89e-53843aadef8d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.827709 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.827683 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:04.976559 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:04.976448 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf"] Jul 16 20:39:04.978728 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:39:04.978701 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14b7f24_e9ce_4b65_a89e_53843aadef8d.slice/crio-829116792c1a185761c9beac70fa5336f9938cd40a4ed4ae202579fc156e0ee4 WatchSource:0}: Error finding container 829116792c1a185761c9beac70fa5336f9938cd40a4ed4ae202579fc156e0ee4: Status 404 returned error can't find the container with id 829116792c1a185761c9beac70fa5336f9938cd40a4ed4ae202579fc156e0ee4 Jul 16 20:39:05.331797 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:05.331764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" event={"ID":"d14b7f24-e9ce-4b65-a89e-53843aadef8d","Type":"ContainerStarted","Data":"829116792c1a185761c9beac70fa5336f9938cd40a4ed4ae202579fc156e0ee4"} Jul 16 20:39:07.067565 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:07.067523 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 20:39:07.067853 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:07.067624 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 20:39:07.344443 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:07.344366 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" event={"ID":"d14b7f24-e9ce-4b65-a89e-53843aadef8d","Type":"ContainerStarted","Data":"78e944c4cf94ce4edfa71c4590a01a352107ec9e912673d3c1bb2c9a8db8bd9a"} Jul 16 20:39:07.344606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:07.344472 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:39:07.382125 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:07.382080 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" podStartSLOduration=1.295472249 podStartE2EDuration="3.382066554s" totalImagesPullingTime="2.086594305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:07.380749124 +0000 UTC m=+355.222600249" watchObservedRunningTime="2026-07-16 20:39:07.382066554 +0000 UTC m=+355.223917679" Jul 16 20:39:08.302102 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:08.302076 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-649bd55f77-wpxkt" Jul 16 20:39:08.349991 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:39:08.349963 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7vrjf" Jul 16 20:40:11.480911 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.480875 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479"] Jul 16 20:40:11.483409 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.483393 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.485747 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.485727 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 20:40:11.486625 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.486601 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 20:40:11.486719 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.486613 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2rp5w\"" Jul 16 20:40:11.491775 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.491751 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479"] Jul 16 20:40:11.587585 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.587552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.587757 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.587639 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.587757 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.587670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldjp\" (UniqueName: \"kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.688965 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.688927 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.689160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.688977 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldjp\" (UniqueName: \"kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.689160 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.689091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.689409 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.689384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.689409 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.689400 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.699740 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.699720 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldjp\" (UniqueName: \"kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.792370 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.792336 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:11.880955 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.880925 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh"] Jul 16 20:40:11.883717 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.883697 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:11.893560 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.893534 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh"] Jul 16 20:40:11.921979 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.921960 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479"] Jul 16 20:40:11.924317 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:40:11.924294 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e6ea38_7f11_4896_9dd3_7acecc7912da.slice/crio-0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70 WatchSource:0}: Error finding container 0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70: Status 404 returned error can't find the container with id 0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70 Jul 16 20:40:11.991409 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.991378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qml6s\" (UniqueName: \"kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:11.991529 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.991442 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:11.991529 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:11.991473 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.092607 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.092514 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.092607 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.092555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.092770 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.092640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qml6s\" (UniqueName: \"kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.092930 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.092909 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.092977 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.092929 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.101009 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.100989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qml6s\" (UniqueName: \"kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.194690 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.194654 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:12.306950 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.306918 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t"] Jul 16 20:40:12.310540 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.310519 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.319729 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.319423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t"] Jul 16 20:40:12.322223 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.322206 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh"] Jul 16 20:40:12.324513 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:40:12.324491 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73674d0_4ace_4c85_833a_32e97edec1fe.slice/crio-871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488 WatchSource:0}: Error finding container 871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488: Status 404 returned error can't find the container with id 871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488 Jul 16 20:40:12.395137 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.395109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4sf\" (UniqueName: \"kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.395284 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.395178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.395284 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.395219 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.496066 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.496037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.496387 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.496080 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.496387 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.496147 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4sf\" (UniqueName: \"kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.496511 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.496416 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.496511 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.496477 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.505905 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.505886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4sf\" (UniqueName: \"kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.619808 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.619757 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:12.644618 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.644587 2582 generic.go:356] "Generic (PLEG): container finished" podID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerID="2792c78611ff69f0e54edc5384a2ecad69153a8b1d97f76c35e7a2005a4f7f80" exitCode=0 Jul 16 20:40:12.644745 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.644658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" event={"ID":"f73674d0-4ace-4c85-833a-32e97edec1fe","Type":"ContainerDied","Data":"2792c78611ff69f0e54edc5384a2ecad69153a8b1d97f76c35e7a2005a4f7f80"} Jul 16 20:40:12.644745 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.644700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" event={"ID":"f73674d0-4ace-4c85-833a-32e97edec1fe","Type":"ContainerStarted","Data":"871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488"} Jul 16 20:40:12.646304 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.646257 2582 generic.go:356] "Generic (PLEG): container finished" podID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerID="4900686bef22740d001edede29a028be09e881f111bc5242292b172893aa217d" exitCode=0 Jul 16 20:40:12.646390 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.646329 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" event={"ID":"42e6ea38-7f11-4896-9dd3-7acecc7912da","Type":"ContainerDied","Data":"4900686bef22740d001edede29a028be09e881f111bc5242292b172893aa217d"} Jul 16 20:40:12.646390 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.646369 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" event={"ID":"42e6ea38-7f11-4896-9dd3-7acecc7912da","Type":"ContainerStarted","Data":"0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70"} Jul 16 20:40:12.689710 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.687685 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn"] Jul 16 20:40:12.710592 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.707633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn"] Jul 16 20:40:12.710592 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.707755 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.780254 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.780229 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t"] Jul 16 20:40:12.781868 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:40:12.781843 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aedf5aa_f093_4d8c_b259_357f945cc650.slice/crio-8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2 WatchSource:0}: Error finding container 8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2: Status 404 returned error can't find the container with id 8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2 Jul 16 20:40:12.801688 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.801665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bncx8\" (UniqueName: \"kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.801778 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.801759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.801884 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.801859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.903220 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.903183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.903349 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.903268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bncx8\" (UniqueName: \"kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.903404 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.903345 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.903541 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.903521 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.903626 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.903606 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:12.911207 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:12.911187 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bncx8\" (UniqueName: \"kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:13.025136 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.025098 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:13.152754 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.152726 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn"] Jul 16 20:40:13.154488 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:40:13.154458 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895d270b_18d6_498b_9bc0_c56b796f9683.slice/crio-c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66 WatchSource:0}: Error finding container c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66: Status 404 returned error can't find the container with id c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66 Jul 16 20:40:13.653731 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.653697 2582 generic.go:356] "Generic (PLEG): container finished" podID="895d270b-18d6-498b-9bc0-c56b796f9683" containerID="8d2a62dc7964693c66d51846e8c4fe8f1f474bec5d9f80a90040767473d04572" exitCode=0 Jul 16 20:40:13.654092 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.653781 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" event={"ID":"895d270b-18d6-498b-9bc0-c56b796f9683","Type":"ContainerDied","Data":"8d2a62dc7964693c66d51846e8c4fe8f1f474bec5d9f80a90040767473d04572"} Jul 16 20:40:13.654092 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.653830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" event={"ID":"895d270b-18d6-498b-9bc0-c56b796f9683","Type":"ContainerStarted","Data":"c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66"} Jul 16 20:40:13.655607 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.655587 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerID="feb99fdb8dcf11e08874ea8f5ca282719b23da82fee01a14d310d5689598d6ff" exitCode=0 Jul 16 20:40:13.655697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.655648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" event={"ID":"8aedf5aa-f093-4d8c-b259-357f945cc650","Type":"ContainerDied","Data":"feb99fdb8dcf11e08874ea8f5ca282719b23da82fee01a14d310d5689598d6ff"} Jul 16 20:40:13.655697 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:13.655682 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" event={"ID":"8aedf5aa-f093-4d8c-b259-357f945cc650","Type":"ContainerStarted","Data":"8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2"} Jul 16 20:40:14.661999 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:14.661973 2582 generic.go:356] "Generic (PLEG): container finished" podID="895d270b-18d6-498b-9bc0-c56b796f9683" containerID="30dff46babd6ef63a1b94da7e1786a8b83435c1f3c8c5364bb7440145685beb0" exitCode=0 Jul 16 20:40:14.662321 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:14.662070 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" event={"ID":"895d270b-18d6-498b-9bc0-c56b796f9683","Type":"ContainerDied","Data":"30dff46babd6ef63a1b94da7e1786a8b83435c1f3c8c5364bb7440145685beb0"} Jul 16 20:40:14.663734 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:14.663716 2582 generic.go:356] "Generic (PLEG): container finished" podID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerID="86ea85b98a029432953afc25dc87b9e129b70c467ed1bd1c081895f112c42041" exitCode=0 Jul 16 20:40:14.663818 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:14.663750 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" event={"ID":"42e6ea38-7f11-4896-9dd3-7acecc7912da","Type":"ContainerDied","Data":"86ea85b98a029432953afc25dc87b9e129b70c467ed1bd1c081895f112c42041"} Jul 16 20:40:15.675961 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.675928 2582 generic.go:356] "Generic (PLEG): container finished" podID="895d270b-18d6-498b-9bc0-c56b796f9683" containerID="7b6c6a99ed5f03e3a905e5f94a773a4461ba93039db6d3f2f98c90de660c3428" exitCode=0 Jul 16 20:40:15.676395 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.676014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" event={"ID":"895d270b-18d6-498b-9bc0-c56b796f9683","Type":"ContainerDied","Data":"7b6c6a99ed5f03e3a905e5f94a773a4461ba93039db6d3f2f98c90de660c3428"} Jul 16 20:40:15.677958 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.677935 2582 generic.go:356] "Generic (PLEG): container finished" podID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerID="a6e19a0de024b38544bb180044939b2af54d3a60254dbd689f472e0694b24727" exitCode=0 Jul 16 20:40:15.678059 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.678005 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" event={"ID":"42e6ea38-7f11-4896-9dd3-7acecc7912da","Type":"ContainerDied","Data":"a6e19a0de024b38544bb180044939b2af54d3a60254dbd689f472e0694b24727"} Jul 16 20:40:15.679731 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.679707 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerID="2e91e9ad657e8ece61d257009a57b869b90dfeca73041f0c6315dd7c90e54ccc" exitCode=0 Jul 16 20:40:15.679820 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:15.679764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" event={"ID":"8aedf5aa-f093-4d8c-b259-357f945cc650","Type":"ContainerDied","Data":"2e91e9ad657e8ece61d257009a57b869b90dfeca73041f0c6315dd7c90e54ccc"} Jul 16 20:40:16.686244 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.686209 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerID="eef89b19952d998c591e3db4f0fa562872d80eff18c8dd3143b8dbda83e3825b" exitCode=0 Jul 16 20:40:16.686653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.686298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" event={"ID":"8aedf5aa-f093-4d8c-b259-357f945cc650","Type":"ContainerDied","Data":"eef89b19952d998c591e3db4f0fa562872d80eff18c8dd3143b8dbda83e3825b"} Jul 16 20:40:16.817138 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.817107 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:16.837325 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.837302 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util\") pod \"895d270b-18d6-498b-9bc0-c56b796f9683\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " Jul 16 20:40:16.837448 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.837405 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bncx8\" (UniqueName: \"kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8\") pod \"895d270b-18d6-498b-9bc0-c56b796f9683\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " Jul 16 20:40:16.837448 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.837423 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle\") pod \"895d270b-18d6-498b-9bc0-c56b796f9683\" (UID: \"895d270b-18d6-498b-9bc0-c56b796f9683\") " Jul 16 20:40:16.838175 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.838147 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle" (OuterVolumeSpecName: "bundle") pod "895d270b-18d6-498b-9bc0-c56b796f9683" (UID: "895d270b-18d6-498b-9bc0-c56b796f9683"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:16.840021 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.839995 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8" (OuterVolumeSpecName: "kube-api-access-bncx8") pod "895d270b-18d6-498b-9bc0-c56b796f9683" (UID: "895d270b-18d6-498b-9bc0-c56b796f9683"). InnerVolumeSpecName "kube-api-access-bncx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:40:16.843346 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.843325 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util" (OuterVolumeSpecName: "util") pod "895d270b-18d6-498b-9bc0-c56b796f9683" (UID: "895d270b-18d6-498b-9bc0-c56b796f9683"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:16.844412 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.844392 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:16.938292 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938233 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle\") pod \"42e6ea38-7f11-4896-9dd3-7acecc7912da\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " Jul 16 20:40:16.938410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938296 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ldjp\" (UniqueName: \"kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp\") pod \"42e6ea38-7f11-4896-9dd3-7acecc7912da\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " Jul 16 20:40:16.938410 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938343 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util\") pod \"42e6ea38-7f11-4896-9dd3-7acecc7912da\" (UID: \"42e6ea38-7f11-4896-9dd3-7acecc7912da\") " Jul 16 20:40:16.938619 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938600 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bncx8\" (UniqueName: \"kubernetes.io/projected/895d270b-18d6-498b-9bc0-c56b796f9683-kube-api-access-bncx8\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:16.938692 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938622 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:16.938692 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938634 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/895d270b-18d6-498b-9bc0-c56b796f9683-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:16.938752 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.938711 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle" (OuterVolumeSpecName: "bundle") pod "42e6ea38-7f11-4896-9dd3-7acecc7912da" (UID: "42e6ea38-7f11-4896-9dd3-7acecc7912da"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:16.940219 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.940200 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp" (OuterVolumeSpecName: "kube-api-access-4ldjp") pod "42e6ea38-7f11-4896-9dd3-7acecc7912da" (UID: "42e6ea38-7f11-4896-9dd3-7acecc7912da"). InnerVolumeSpecName "kube-api-access-4ldjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:40:16.945606 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:16.945584 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util" (OuterVolumeSpecName: "util") pod "42e6ea38-7f11-4896-9dd3-7acecc7912da" (UID: "42e6ea38-7f11-4896-9dd3-7acecc7912da"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:17.039868 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.039840 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:17.039868 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.039866 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ldjp\" (UniqueName: \"kubernetes.io/projected/42e6ea38-7f11-4896-9dd3-7acecc7912da-kube-api-access-4ldjp\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:17.040011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.039876 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e6ea38-7f11-4896-9dd3-7acecc7912da-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:17.696269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.696242 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" Jul 16 20:40:17.696269 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.696261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn" event={"ID":"895d270b-18d6-498b-9bc0-c56b796f9683","Type":"ContainerDied","Data":"c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66"} Jul 16 20:40:17.696817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.696307 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c53242360c570e5abbf4306678b32920370d5a8182e207b9b9b83af2c7e3de66" Jul 16 20:40:17.698042 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.698022 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" Jul 16 20:40:17.698163 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.698062 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479" event={"ID":"42e6ea38-7f11-4896-9dd3-7acecc7912da","Type":"ContainerDied","Data":"0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70"} Jul 16 20:40:17.698163 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.698098 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bf4af439b6726fbcc9a82e32fb6b1fa33e5befe0ae857f3c797ef341aad0e70" Jul 16 20:40:17.834448 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.834424 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:17.946180 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.946138 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util\") pod \"8aedf5aa-f093-4d8c-b259-357f945cc650\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " Jul 16 20:40:17.946180 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.946187 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg4sf\" (UniqueName: \"kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf\") pod \"8aedf5aa-f093-4d8c-b259-357f945cc650\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " Jul 16 20:40:17.946448 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.946213 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle\") pod \"8aedf5aa-f093-4d8c-b259-357f945cc650\" (UID: \"8aedf5aa-f093-4d8c-b259-357f945cc650\") " Jul 16 20:40:17.946653 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.946626 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle" (OuterVolumeSpecName: "bundle") pod "8aedf5aa-f093-4d8c-b259-357f945cc650" (UID: "8aedf5aa-f093-4d8c-b259-357f945cc650"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:17.948131 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.948105 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf" (OuterVolumeSpecName: "kube-api-access-xg4sf") pod "8aedf5aa-f093-4d8c-b259-357f945cc650" (UID: "8aedf5aa-f093-4d8c-b259-357f945cc650"). InnerVolumeSpecName "kube-api-access-xg4sf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:40:17.951475 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:17.951453 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util" (OuterVolumeSpecName: "util") pod "8aedf5aa-f093-4d8c-b259-357f945cc650" (UID: "8aedf5aa-f093-4d8c-b259-357f945cc650"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:18.047004 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.046962 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg4sf\" (UniqueName: \"kubernetes.io/projected/8aedf5aa-f093-4d8c-b259-357f945cc650-kube-api-access-xg4sf\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:18.047004 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.046998 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:18.047004 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.047008 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aedf5aa-f093-4d8c-b259-357f945cc650-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:18.704392 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.704317 2582 generic.go:356] "Generic (PLEG): container finished" podID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerID="87030ca0550c6886f574c246fa43c8c00c7606fc00b10cfd0ccc1748fc0f6a18" exitCode=0 Jul 16 20:40:18.704771 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.704396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" event={"ID":"f73674d0-4ace-4c85-833a-32e97edec1fe","Type":"ContainerDied","Data":"87030ca0550c6886f574c246fa43c8c00c7606fc00b10cfd0ccc1748fc0f6a18"} Jul 16 20:40:18.706389 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.706369 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" event={"ID":"8aedf5aa-f093-4d8c-b259-357f945cc650","Type":"ContainerDied","Data":"8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2"} Jul 16 20:40:18.706458 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.706400 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t" Jul 16 20:40:18.706497 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:18.706404 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e8a2cc5d82efe945cee9e3d79806c3ac2069d88c9518d52bbf8a49eb902c2f2" Jul 16 20:40:19.713714 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:19.713678 2582 generic.go:356] "Generic (PLEG): container finished" podID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerID="af832168341f87aa727095ce4782516d40e7702bf46fb84b2683f58dd4ac0551" exitCode=0 Jul 16 20:40:19.714081 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:19.713769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" event={"ID":"f73674d0-4ace-4c85-833a-32e97edec1fe","Type":"ContainerDied","Data":"af832168341f87aa727095ce4782516d40e7702bf46fb84b2683f58dd4ac0551"} Jul 16 20:40:20.844314 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.844292 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:20.868911 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.868884 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle\") pod \"f73674d0-4ace-4c85-833a-32e97edec1fe\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " Jul 16 20:40:20.869043 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.868958 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qml6s\" (UniqueName: \"kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s\") pod \"f73674d0-4ace-4c85-833a-32e97edec1fe\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " Jul 16 20:40:20.869043 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.869008 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util\") pod \"f73674d0-4ace-4c85-833a-32e97edec1fe\" (UID: \"f73674d0-4ace-4c85-833a-32e97edec1fe\") " Jul 16 20:40:20.869717 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.869684 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle" (OuterVolumeSpecName: "bundle") pod "f73674d0-4ace-4c85-833a-32e97edec1fe" (UID: "f73674d0-4ace-4c85-833a-32e97edec1fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:20.871042 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.871015 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s" (OuterVolumeSpecName: "kube-api-access-qml6s") pod "f73674d0-4ace-4c85-833a-32e97edec1fe" (UID: "f73674d0-4ace-4c85-833a-32e97edec1fe"). InnerVolumeSpecName "kube-api-access-qml6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:40:20.873468 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.873433 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util" (OuterVolumeSpecName: "util") pod "f73674d0-4ace-4c85-833a-32e97edec1fe" (UID: "f73674d0-4ace-4c85-833a-32e97edec1fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 20:40:20.969784 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.969751 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:20.969784 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.969785 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73674d0-4ace-4c85-833a-32e97edec1fe-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:20.969903 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:20.969799 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qml6s\" (UniqueName: \"kubernetes.io/projected/f73674d0-4ace-4c85-833a-32e97edec1fe-kube-api-access-qml6s\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:40:21.727011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:21.726969 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" event={"ID":"f73674d0-4ace-4c85-833a-32e97edec1fe","Type":"ContainerDied","Data":"871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488"} Jul 16 20:40:21.727011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:21.726998 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh" Jul 16 20:40:21.727011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:40:21.727011 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="871bd4c642e8533e10ae484956c10c59a410dff016570faa5250840f2da0d488" Jul 16 20:41:20.627956 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.627920 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628219 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628230 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628240 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628246 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628254 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628260 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628269 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628274 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628280 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628286 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628291 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628296 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628300 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628305 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628310 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628315 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628321 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628326 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="util" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628332 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628337 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628343 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628348 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="pull" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628358 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628363 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628413 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aedf5aa-f093-4d8c-b259-357f945cc650" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628422 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="895d270b-18d6-498b-9bc0-c56b796f9683" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628428 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73674d0-4ace-4c85-833a-32e97edec1fe" containerName="extract" Jul 16 20:41:20.628424 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.628436 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e6ea38-7f11-4896-9dd3-7acecc7912da" containerName="extract" Jul 16 20:41:20.631307 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.631292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.633567 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.633547 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 20:41:20.633698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.633644 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 20:41:20.633698 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.633661 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2rp5w\"" Jul 16 20:41:20.634463 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.634450 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 20:41:20.638323 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.638119 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:41:20.658936 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.658912 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:41:20.720124 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.720094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zxw\" (UniqueName: \"kubernetes.io/projected/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-kube-api-access-54zxw\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.720236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.720131 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-config-file\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.821094 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.821067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54zxw\" (UniqueName: \"kubernetes.io/projected/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-kube-api-access-54zxw\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.821226 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.821109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-config-file\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.821772 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.821750 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-config-file\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.830417 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.830398 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zxw\" (UniqueName: \"kubernetes.io/projected/224fae5f-8d3c-4711-aaa2-f2317f0be9fb-kube-api-access-54zxw\") pod \"limitador-limitador-54d6d988d5-4b72t\" (UID: \"224fae5f-8d3c-4711-aaa2-f2317f0be9fb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:20.942719 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:20.942673 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:21.068337 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:21.068307 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:41:21.069435 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:41:21.069410 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224fae5f_8d3c_4711_aaa2_f2317f0be9fb.slice/crio-279c458c9b2352045ea0b91ac04a5f5b410cf001fe51eb20d0a642f1ba42c5d9 WatchSource:0}: Error finding container 279c458c9b2352045ea0b91ac04a5f5b410cf001fe51eb20d0a642f1ba42c5d9: Status 404 returned error can't find the container with id 279c458c9b2352045ea0b91ac04a5f5b410cf001fe51eb20d0a642f1ba42c5d9 Jul 16 20:41:22.022888 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:22.022845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" event={"ID":"224fae5f-8d3c-4711-aaa2-f2317f0be9fb","Type":"ContainerStarted","Data":"279c458c9b2352045ea0b91ac04a5f5b410cf001fe51eb20d0a642f1ba42c5d9"} Jul 16 20:41:24.035662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.035630 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" event={"ID":"224fae5f-8d3c-4711-aaa2-f2317f0be9fb","Type":"ContainerStarted","Data":"04abf30e8a9e606f1a8250946c13a54138b7accb18c1b00a1a0b1a85c8053ef3"} Jul 16 20:41:24.036062 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.035812 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:24.052119 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.052048 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" podStartSLOduration=1.41138342 podStartE2EDuration="4.052032424s" totalImagesPullingTime="2.640649004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:24.051754427 +0000 UTC m=+491.893605554" watchObservedRunningTime="2026-07-16 20:41:24.052032424 +0000 UTC m=+491.893883550" Jul 16 20:41:24.706328 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.706296 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d4c586979-m7r8b"] Jul 16 20:41:24.709951 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.709926 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.725084 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.725058 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4c586979-m7r8b"] Jul 16 20:41:24.754123 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-console-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754236 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-oauth-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754281 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754228 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-oauth-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754281 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87wd9\" (UniqueName: \"kubernetes.io/projected/96307a7f-010b-4845-8fab-c7c196623fbb-kube-api-access-87wd9\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754342 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754290 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-trusted-ca-bundle\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754386 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754372 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-service-ca\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.754419 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.754400 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.854956 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.854923 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-console-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.854987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-oauth-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-oauth-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855043 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87wd9\" (UniqueName: \"kubernetes.io/projected/96307a7f-010b-4845-8fab-c7c196623fbb-kube-api-access-87wd9\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855120 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-trusted-ca-bundle\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-service-ca\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855324 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855719 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855690 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-console-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855822 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-oauth-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.855940 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855915 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-service-ca\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.856003 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.855926 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96307a7f-010b-4845-8fab-c7c196623fbb-trusted-ca-bundle\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.857345 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.857325 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-oauth-config\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.857471 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.857452 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96307a7f-010b-4845-8fab-c7c196623fbb-console-serving-cert\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:24.862542 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:24.862521 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87wd9\" (UniqueName: \"kubernetes.io/projected/96307a7f-010b-4845-8fab-c7c196623fbb-kube-api-access-87wd9\") pod \"console-d4c586979-m7r8b\" (UID: \"96307a7f-010b-4845-8fab-c7c196623fbb\") " pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:25.020359 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:25.020296 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:25.150662 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:25.150633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4c586979-m7r8b"] Jul 16 20:41:25.152521 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:41:25.152498 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96307a7f_010b_4845_8fab_c7c196623fbb.slice/crio-9742017f68f0a2d1fb2422a10d118f3fc6d4d5077f8f8e9db51ad915ec5a9261 WatchSource:0}: Error finding container 9742017f68f0a2d1fb2422a10d118f3fc6d4d5077f8f8e9db51ad915ec5a9261: Status 404 returned error can't find the container with id 9742017f68f0a2d1fb2422a10d118f3fc6d4d5077f8f8e9db51ad915ec5a9261 Jul 16 20:41:26.047521 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:26.047487 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c586979-m7r8b" event={"ID":"96307a7f-010b-4845-8fab-c7c196623fbb","Type":"ContainerStarted","Data":"fd723259814d5a73eaaa64dbde5c1f2faa6a6e7c5833404df8bb6b4e82d43262"} Jul 16 20:41:26.047810 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:26.047535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c586979-m7r8b" event={"ID":"96307a7f-010b-4845-8fab-c7c196623fbb","Type":"ContainerStarted","Data":"9742017f68f0a2d1fb2422a10d118f3fc6d4d5077f8f8e9db51ad915ec5a9261"} Jul 16 20:41:26.068178 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:26.068136 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d4c586979-m7r8b" podStartSLOduration=2.068122374 podStartE2EDuration="2.068122374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:26.065038451 +0000 UTC m=+493.906889579" watchObservedRunningTime="2026-07-16 20:41:26.068122374 +0000 UTC m=+493.909973499" Jul 16 20:41:35.020650 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.020618 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:35.021027 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.020673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:35.025862 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.025839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:35.041916 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.041894 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4b72t" Jul 16 20:41:35.095804 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.095779 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d4c586979-m7r8b" Jul 16 20:41:35.146592 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:41:35.146553 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:42:00.175234 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.175191 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c8b549575-xbjps" podUID="15225327-1798-425c-b9bc-dc6f8f35d95d" containerName="console" containerID="cri-o://b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4" gracePeriod=15 Jul 16 20:42:00.439302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.439277 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:42:00.534393 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534370 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534501 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534421 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534501 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534450 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534501 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534479 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-565w2\" (UniqueName: \"kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534650 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534500 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534650 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534632 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534754 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534703 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert\") pod \"15225327-1798-425c-b9bc-dc6f8f35d95d\" (UID: \"15225327-1798-425c-b9bc-dc6f8f35d95d\") " Jul 16 20:42:00.534894 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534847 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca" (OuterVolumeSpecName: "service-ca") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:42:00.534894 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534861 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config" (OuterVolumeSpecName: "console-config") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:42:00.535121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534953 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:42:00.535121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.534998 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-console-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.535121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.535018 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.535121 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.535071 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:42:00.536519 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.536500 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:42:00.537035 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.536997 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 20:42:00.537035 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.537012 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2" (OuterVolumeSpecName: "kube-api-access-565w2") pod "15225327-1798-425c-b9bc-dc6f8f35d95d" (UID: "15225327-1798-425c-b9bc-dc6f8f35d95d"). InnerVolumeSpecName "kube-api-access-565w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 20:42:00.635794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.635771 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-oauth-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.635794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.635791 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-oauth-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.635916 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.635800 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15225327-1798-425c-b9bc-dc6f8f35d95d-console-serving-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.635916 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.635809 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-565w2\" (UniqueName: \"kubernetes.io/projected/15225327-1798-425c-b9bc-dc6f8f35d95d-kube-api-access-565w2\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:00.635916 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:00.635818 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15225327-1798-425c-b9bc-dc6f8f35d95d-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:42:01.219115 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.219083 2582 generic.go:356] "Generic (PLEG): container finished" podID="15225327-1798-425c-b9bc-dc6f8f35d95d" containerID="b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4" exitCode=0 Jul 16 20:42:01.219505 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.219123 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8b549575-xbjps" Jul 16 20:42:01.219505 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.219164 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8b549575-xbjps" event={"ID":"15225327-1798-425c-b9bc-dc6f8f35d95d","Type":"ContainerDied","Data":"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4"} Jul 16 20:42:01.219505 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.219217 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8b549575-xbjps" event={"ID":"15225327-1798-425c-b9bc-dc6f8f35d95d","Type":"ContainerDied","Data":"c3b340057588edfb597818315f7af5d546bb0123f8616e78b8fa5ea8950d62b0"} Jul 16 20:42:01.219505 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.219243 2582 scope.go:117] "RemoveContainer" containerID="b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4" Jul 16 20:42:01.231761 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.231663 2582 scope.go:117] "RemoveContainer" containerID="b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4" Jul 16 20:42:01.232453 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:42:01.232425 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4\": container with ID starting with b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4 not found: ID does not exist" containerID="b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4" Jul 16 20:42:01.232551 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.232460 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4"} err="failed to get container status \"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4\": rpc error: code = NotFound desc = could not find container \"b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4\": container with ID starting with b381b555e7d7d101c7f8f78ea2bb5cbeb4cb25635ef065e777cb5406905568f4 not found: ID does not exist" Jul 16 20:42:01.236020 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.235996 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:42:01.239885 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:01.239861 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c8b549575-xbjps"] Jul 16 20:42:02.721877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:02.721847 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15225327-1798-425c-b9bc-dc6f8f35d95d" path="/var/lib/kubelet/pods/15225327-1798-425c-b9bc-dc6f8f35d95d/volumes" Jul 16 20:42:29.967983 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:29.967945 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:42:53.050078 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:53.050040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:42:59.932707 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:42:59.932667 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:43:12.138714 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:12.138680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:43:12.639850 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:12.639822 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:43:12.640298 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:12.640203 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:43:12.643871 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:12.643853 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:43:12.644329 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:12.644304 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:43:18.034467 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:18.034426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:43:21.987069 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.987033 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5"] Jul 16 20:43:21.988921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.987362 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15225327-1798-425c-b9bc-dc6f8f35d95d" containerName="console" Jul 16 20:43:21.988921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.987373 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="15225327-1798-425c-b9bc-dc6f8f35d95d" containerName="console" Jul 16 20:43:21.988921 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.987437 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="15225327-1798-425c-b9bc-dc6f8f35d95d" containerName="console" Jul 16 20:43:21.989355 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.989340 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:21.991936 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.991901 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 20:43:21.992066 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.991998 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 20:43:21.993019 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.992997 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 20:43:21.993135 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:21.993077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jqnfp\"" Jul 16 20:43:22.001564 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.001542 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5"] Jul 16 20:43:22.056983 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.056952 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.056983 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.056988 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.057177 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.057079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6cxk\" (UniqueName: \"kubernetes.io/projected/21abb431-60d7-4139-ada2-05bfbd36d4a0-kube-api-access-h6cxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.057177 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.057155 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.057256 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.057179 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.057256 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.057208 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21abb431-60d7-4139-ada2-05bfbd36d4a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158285 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6cxk\" (UniqueName: \"kubernetes.io/projected/21abb431-60d7-4139-ada2-05bfbd36d4a0-kube-api-access-h6cxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21abb431-60d7-4139-ada2-05bfbd36d4a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158454 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158777 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158894 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.158894 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.158825 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.160594 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.160557 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21abb431-60d7-4139-ada2-05bfbd36d4a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.160773 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.160756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21abb431-60d7-4139-ada2-05bfbd36d4a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.166765 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.166741 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6cxk\" (UniqueName: \"kubernetes.io/projected/21abb431-60d7-4139-ada2-05bfbd36d4a0-kube-api-access-h6cxk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hbwp5\" (UID: \"21abb431-60d7-4139-ada2-05bfbd36d4a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.300076 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.300051 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:22.424879 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.424850 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5"] Jul 16 20:43:22.426189 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:43:22.426163 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21abb431_60d7_4139_ada2_05bfbd36d4a0.slice/crio-764aa4c1c1e0cbb9ba3244143ad8bb77e2141aeddfe507198cafdf2e77c7d8f3 WatchSource:0}: Error finding container 764aa4c1c1e0cbb9ba3244143ad8bb77e2141aeddfe507198cafdf2e77c7d8f3: Status 404 returned error can't find the container with id 764aa4c1c1e0cbb9ba3244143ad8bb77e2141aeddfe507198cafdf2e77c7d8f3 Jul 16 20:43:22.427548 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.427525 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 20:43:22.619485 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:22.619398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" event={"ID":"21abb431-60d7-4139-ada2-05bfbd36d4a0","Type":"ContainerStarted","Data":"764aa4c1c1e0cbb9ba3244143ad8bb77e2141aeddfe507198cafdf2e77c7d8f3"} Jul 16 20:43:23.624850 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:23.624823 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:43:23.626705 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:23.626673 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" event={"ID":"21abb431-60d7-4139-ada2-05bfbd36d4a0","Type":"ContainerStarted","Data":"219e407f53c2f241c981eff9315df140286be312ac368142d269577c9b3b004b"} Jul 16 20:43:23.644381 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:23.644313 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" podStartSLOduration=1.639691912 podStartE2EDuration="2.64429254s" totalImagesPullingTime="1.004600628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:43:23.644212051 +0000 UTC m=+611.486063172" watchObservedRunningTime="2026-07-16 20:43:23.64429254 +0000 UTC m=+611.486143666" Jul 16 20:43:32.300658 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:32.300619 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:32.301056 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:32.300690 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:32.312906 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:32.312885 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:32.684053 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:32.683969 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hbwp5" Jul 16 20:43:36.449996 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:43:36.449918 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:44:48.937622 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:44:48.937587 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:44:59.035943 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:44:59.035907 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:45:00.163794 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.163764 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp"] Jul 16 20:45:00.167022 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.167007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" Jul 16 20:45:00.170038 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.170021 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 20:45:00.175886 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.175868 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" Jul 16 20:45:00.183006 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.182983 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp"] Jul 16 20:45:00.305969 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:00.305946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp"] Jul 16 20:45:00.307680 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:45:00.307653 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4c89b1_49b3_46fd_95dc_4883d67127f1.slice/crio-7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed WatchSource:0}: Error finding container 7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed: Status 404 returned error can't find the container with id 7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed Jul 16 20:45:01.122481 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:01.122446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" event={"ID":"fd4c89b1-49b3-46fd-95dc-4883d67127f1","Type":"ContainerStarted","Data":"7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed"} Jul 16 20:45:03.135625 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:03.135594 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" event={"ID":"fd4c89b1-49b3-46fd-95dc-4883d67127f1","Type":"ContainerStarted","Data":"13d1d0abc951fb5fbe34fd48cbade2bf5d9697bc54afe309b65a70b7bdb9b54b"} Jul 16 20:45:04.142750 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:04.142663 2582 generic.go:356] "Generic (PLEG): container finished" podID="fd4c89b1-49b3-46fd-95dc-4883d67127f1" containerID="13d1d0abc951fb5fbe34fd48cbade2bf5d9697bc54afe309b65a70b7bdb9b54b" exitCode=0 Jul 16 20:45:04.143109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:04.142747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" event={"ID":"fd4c89b1-49b3-46fd-95dc-4883d67127f1","Type":"ContainerDied","Data":"13d1d0abc951fb5fbe34fd48cbade2bf5d9697bc54afe309b65a70b7bdb9b54b"} Jul 16 20:45:05.274876 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:05.274853 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" Jul 16 20:45:06.154139 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:06.154112 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" Jul 16 20:45:06.154306 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:06.154155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-pwmkp" event={"ID":"fd4c89b1-49b3-46fd-95dc-4883d67127f1","Type":"ContainerDied","Data":"7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed"} Jul 16 20:45:06.154306 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:06.154201 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d77c51153dbec7c379749e7d6a245cb5c19d0d26467d0e0f91c0988459423ed" Jul 16 20:45:09.551322 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:45:09.551292 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:46:50.629634 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:46:50.629547 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:46:59.830877 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:46:59.830839 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:47:31.437431 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:47:31.437391 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:47:41.034217 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:47:41.034179 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:47:51.941543 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:47:51.941511 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:48:00.429646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:00.429612 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:48:12.036522 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:12.036444 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:48:12.672820 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:12.672790 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:48:12.675456 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:12.675432 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:48:12.677355 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:12.677335 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:48:12.680028 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:12.680012 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:48:22.241109 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:22.241074 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:48:34.429176 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:34.429143 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:48:44.335103 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:48:44.335062 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:49:09.542945 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:49:09.542907 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:49:20.237302 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:49:20.237264 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:49:43.634369 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:49:43.634289 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:49:57.228967 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:49:57.228927 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:50:13.330853 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:50:13.330822 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:50:22.640912 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:50:22.640880 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:50:39.634104 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:50:39.634067 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:52:40.435673 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:52:40.435583 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:52:52.130635 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:52:52.130589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:01.042270 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:01.042206 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:12.706922 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:12.706894 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:53:12.711145 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:12.711120 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:53:12.712014 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:12.711996 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:53:12.716027 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:12.716010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:53:15.839785 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:15.839752 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:25.633943 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:25.633904 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:34.845225 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:34.845186 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:43.933911 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:43.933867 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:53:54.822198 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:53:54.822141 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:54:05.429639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:54:05.429536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:54:14.135516 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:54:14.135481 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:54:25.532026 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:54:25.531987 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:54:47.368836 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:54:47.368796 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:54:56.735962 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:54:56.735931 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:55:07.144220 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:55:07.144171 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:55:17.142920 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:55:17.142880 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:55:29.528391 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:55:29.528354 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:55:43.740639 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:55:43.740539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:55:56.631591 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:55:56.631538 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:56:06.072646 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:56:06.072609 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:56:19.740957 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:56:19.740913 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:56:29.740190 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:56:29.740146 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:56:39.927587 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:56:39.927544 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:56:51.734861 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:56:51.734823 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:57:07.232155 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:57:07.232072 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:57:15.433925 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:57:15.433876 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:57:26.934126 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:57:26.934090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:57:45.832068 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:57:45.832029 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:57:58.736856 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:57:58.736818 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:58:10.134607 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:10.131975 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:58:12.740860 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:12.740830 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:58:12.745399 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:12.745368 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:58:12.754563 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:12.754531 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 20:58:12.758558 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:12.758536 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 20:58:19.242905 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:19.242870 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:58:32.240175 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:32.240134 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:58:46.726827 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:46.726789 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:58:52.426714 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.426668 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:58:52.427139 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.427061 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4c89b1-49b3-46fd-95dc-4883d67127f1" containerName="cleanup" Jul 16 20:58:52.427139 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.427077 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4c89b1-49b3-46fd-95dc-4883d67127f1" containerName="cleanup" Jul 16 20:58:52.427230 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.427162 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4c89b1-49b3-46fd-95dc-4883d67127f1" containerName="cleanup" Jul 16 20:58:52.430136 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.430117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:58:52.432890 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.432866 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 20:58:52.433036 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.432914 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 20:58:52.438727 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.438692 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:58:52.474804 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.474765 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-7sg5z\" (UID: \"befc57a9-591e-4772-86c9-fe16ca2d6049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:58:52.575937 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.575888 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-7sg5z\" (UID: \"befc57a9-591e-4772-86c9-fe16ca2d6049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:58:52.576490 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.576469 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-7sg5z\" (UID: \"befc57a9-591e-4772-86c9-fe16ca2d6049\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:58:52.742837 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.742743 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:58:52.886373 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:52.886349 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:58:52.888407 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:58:52.888377 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbefc57a9_591e_4772_86c9_fe16ca2d6049.slice/crio-d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427 WatchSource:0}: Error finding container d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427: Status 404 returned error can't find the container with id d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427 Jul 16 20:58:53.454049 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:53.453958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerStarted","Data":"af682afa504fdd7db2685a38466af78e49a144bf656e6cb3065b692c0be57fe1"} Jul 16 20:58:53.454049 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:53.454019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerStarted","Data":"d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427"} Jul 16 20:58:53.471157 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:53.471084 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" podStartSLOduration=1.471065663 podStartE2EDuration="1.471065663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:58:53.470237026 +0000 UTC m=+1541.312088155" watchObservedRunningTime="2026-07-16 20:58:53.471065663 +0000 UTC m=+1541.312916791" Jul 16 20:58:54.460843 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:54.460803 2582 generic.go:356] "Generic (PLEG): container finished" podID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerID="af682afa504fdd7db2685a38466af78e49a144bf656e6cb3065b692c0be57fe1" exitCode=7 Jul 16 20:58:54.461270 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:54.460895 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerDied","Data":"af682afa504fdd7db2685a38466af78e49a144bf656e6cb3065b692c0be57fe1"} Jul 16 20:58:54.461270 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:54.461069 2582 scope.go:117] "RemoveContainer" containerID="af682afa504fdd7db2685a38466af78e49a144bf656e6cb3065b692c0be57fe1" Jul 16 20:58:55.468567 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:55.468532 2582 generic.go:356] "Generic (PLEG): container finished" podID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerID="1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c" exitCode=7 Jul 16 20:58:55.469090 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:55.468606 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerDied","Data":"1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c"} Jul 16 20:58:55.469090 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:55.468671 2582 scope.go:117] "RemoveContainer" containerID="af682afa504fdd7db2685a38466af78e49a144bf656e6cb3065b692c0be57fe1" Jul 16 20:58:55.469090 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:55.468895 2582 scope.go:117] "RemoveContainer" containerID="1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c" Jul 16 20:58:55.469257 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:58:55.469164 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-866d7766-7sg5z_odh-ai-gateway-infra(befc57a9-591e-4772-86c9-fe16ca2d6049)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" Jul 16 20:58:56.476063 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:56.475983 2582 scope.go:117] "RemoveContainer" containerID="1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c" Jul 16 20:58:56.476416 ip-10-0-135-240 kubenswrapper[2582]: E0716 20:58:56.476183 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-866d7766-7sg5z_odh-ai-gateway-infra(befc57a9-591e-4772-86c9-fe16ca2d6049)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" Jul 16 20:58:58.627986 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:58:58.627943 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 20:59:08.717004 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:08.716968 2582 scope.go:117] "RemoveContainer" containerID="1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c" Jul 16 20:59:09.546442 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:09.546403 2582 generic.go:356] "Generic (PLEG): container finished" podID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerID="94dadd376a62d8d759df4c7b6632fe61a59eea09d5b0dbd89c2076f8174b4f3b" exitCode=0 Jul 16 20:59:09.546634 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:09.546472 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerDied","Data":"94dadd376a62d8d759df4c7b6632fe61a59eea09d5b0dbd89c2076f8174b4f3b"} Jul 16 20:59:09.546634 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:09.546541 2582 scope.go:117] "RemoveContainer" containerID="1d193a0d282ed0e6a0be3119104b2d0a21622e787b7b5d766a23df953f9d0e4c" Jul 16 20:59:10.580545 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:10.580498 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:59:10.693852 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:10.693822 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:59:10.837399 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:10.837297 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca\") pod \"befc57a9-591e-4772-86c9-fe16ca2d6049\" (UID: \"befc57a9-591e-4772-86c9-fe16ca2d6049\") " Jul 16 20:59:10.837718 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:10.837695 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "befc57a9-591e-4772-86c9-fe16ca2d6049" (UID: "befc57a9-591e-4772-86c9-fe16ca2d6049"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:59:10.938892 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:10.938854 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/befc57a9-591e-4772-86c9-fe16ca2d6049-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:59:11.561011 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:11.560974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" event={"ID":"befc57a9-591e-4772-86c9-fe16ca2d6049","Type":"ContainerDied","Data":"d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427"} Jul 16 20:59:11.561199 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:11.561025 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d67c3fa25455dc7620b346f09982a24bde4144dd8ea07f2a0268f9d2bcd7f427" Jul 16 20:59:11.561199 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:11.560985 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z" Jul 16 20:59:11.590858 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:11.590821 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:59:11.598603 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:11.598549 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-7sg5z"] Jul 16 20:59:12.724357 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:12.724321 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" path="/var/lib/kubelet/pods/befc57a9-591e-4772-86c9-fe16ca2d6049/volumes" Jul 16 20:59:42.468846 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.468802 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx"] Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469159 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469170 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469180 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469186 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469194 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469200 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469291 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.469379 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.469298 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 20:59:42.472584 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.472555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:42.475327 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.475302 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 20:59:42.475455 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.475359 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 20:59:42.479736 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.479703 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx"] Jul 16 20:59:42.515609 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.515558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-svhhx\" (UID: \"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:42.616709 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.616663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-svhhx\" (UID: \"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:42.617332 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.617311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-866d7766-svhhx\" (UID: \"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:42.784817 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.784780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:42.932439 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:42.932397 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx"] Jul 16 20:59:42.934973 ip-10-0-135-240 kubenswrapper[2582]: W0716 20:59:42.934946 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4baf2db1_91dc_4c5b_b211_b1e1b0b94a35.slice/crio-ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423 WatchSource:0}: Error finding container ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423: Status 404 returned error can't find the container with id ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423 Jul 16 20:59:43.733630 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:43.733590 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" event={"ID":"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35","Type":"ContainerStarted","Data":"4c628adfc4b303cbd83f7056028ecb74dc7f4ed56f6ecf0cac311c9c39290609"} Jul 16 20:59:43.734022 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:43.733658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" event={"ID":"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35","Type":"ContainerStarted","Data":"ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423"} Jul 16 20:59:44.740493 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:44.740459 2582 generic.go:356] "Generic (PLEG): container finished" podID="4baf2db1-91dc-4c5b-b211-b1e1b0b94a35" containerID="4c628adfc4b303cbd83f7056028ecb74dc7f4ed56f6ecf0cac311c9c39290609" exitCode=0 Jul 16 20:59:44.740971 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:44.740535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" event={"ID":"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35","Type":"ContainerDied","Data":"4c628adfc4b303cbd83f7056028ecb74dc7f4ed56f6ecf0cac311c9c39290609"} Jul 16 20:59:44.881540 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:44.881511 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:44.939941 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:44.939902 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca\") pod \"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35\" (UID: \"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35\") " Jul 16 20:59:44.940312 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:44.940289 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4baf2db1-91dc-4c5b-b211-b1e1b0b94a35" (UID: "4baf2db1-91dc-4c5b-b211-b1e1b0b94a35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 20:59:45.041083 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:45.041052 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf2db1-91dc-4c5b-b211-b1e1b0b94a35-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 20:59:45.748380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:45.748344 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" Jul 16 20:59:45.748380 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:45.748368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-866d7766-svhhx" event={"ID":"4baf2db1-91dc-4c5b-b211-b1e1b0b94a35","Type":"ContainerDied","Data":"ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423"} Jul 16 20:59:45.748871 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:45.748416 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba67e98a2aff237a3d229f06efd43283f02e9014573fac932e2f36fcd918c423" Jul 16 20:59:55.426745 ip-10-0-135-240 kubenswrapper[2582]: I0716 20:59:55.426707 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:00:00.168314 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.168273 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm"] Jul 16 21:00:00.168744 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.168631 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4baf2db1-91dc-4c5b-b211-b1e1b0b94a35" containerName="revoke-keys" Jul 16 21:00:00.168744 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.168642 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4baf2db1-91dc-4c5b-b211-b1e1b0b94a35" containerName="revoke-keys" Jul 16 21:00:00.168744 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.168707 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="befc57a9-591e-4772-86c9-fe16ca2d6049" containerName="revoke-keys" Jul 16 21:00:00.168744 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.168716 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4baf2db1-91dc-4c5b-b211-b1e1b0b94a35" containerName="revoke-keys" Jul 16 21:00:00.171915 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.171897 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" Jul 16 21:00:00.176723 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.176700 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:00:00.181134 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.181114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" Jul 16 21:00:00.194965 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.194930 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm"] Jul 16 21:00:00.340684 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.340650 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm"] Jul 16 21:00:00.342077 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:00:00.342042 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273f2cf5_413c_43a2_9108_6d4548e58334.slice/crio-f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329 WatchSource:0}: Error finding container f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329: Status 404 returned error can't find the container with id f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329 Jul 16 21:00:00.828588 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.828534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" event={"ID":"273f2cf5-413c-43a2-9108-6d4548e58334","Type":"ContainerStarted","Data":"3086db2518d45cdce90d3144e7149aea3215c2bf71ae6a5706c2d0ecd8a37e19"} Jul 16 21:00:00.828787 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.828606 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" event={"ID":"273f2cf5-413c-43a2-9108-6d4548e58334","Type":"ContainerStarted","Data":"f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329"} Jul 16 21:00:00.846350 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:00.846295 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" podStartSLOduration=0.846276189 podStartE2EDuration="846.276189ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:00:00.844126319 +0000 UTC m=+1608.685977447" watchObservedRunningTime="2026-07-16 21:00:00.846276189 +0000 UTC m=+1608.688127294" Jul 16 21:00:01.835283 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:01.835244 2582 generic.go:356] "Generic (PLEG): container finished" podID="273f2cf5-413c-43a2-9108-6d4548e58334" containerID="3086db2518d45cdce90d3144e7149aea3215c2bf71ae6a5706c2d0ecd8a37e19" exitCode=0 Jul 16 21:00:01.835767 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:01.835332 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" event={"ID":"273f2cf5-413c-43a2-9108-6d4548e58334","Type":"ContainerDied","Data":"3086db2518d45cdce90d3144e7149aea3215c2bf71ae6a5706c2d0ecd8a37e19"} Jul 16 21:00:02.976144 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:02.976122 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" Jul 16 21:00:03.846961 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:03.846933 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" Jul 16 21:00:03.847123 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:03.846925 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-sfbmm" event={"ID":"273f2cf5-413c-43a2-9108-6d4548e58334","Type":"ContainerDied","Data":"f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329"} Jul 16 21:00:03.847123 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:03.847054 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9c4a307ce7d5f36ad7078f646ecb7ed6eb302b6dce3b0b31b8970257f3c2329" Jul 16 21:00:09.840468 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.840427 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:09.840902 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.840813 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273f2cf5-413c-43a2-9108-6d4548e58334" containerName="cleanup" Jul 16 21:00:09.840902 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.840825 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="273f2cf5-413c-43a2-9108-6d4548e58334" containerName="cleanup" Jul 16 21:00:09.840902 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.840882 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="273f2cf5-413c-43a2-9108-6d4548e58334" containerName="cleanup" Jul 16 21:00:09.843852 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.843834 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:09.846487 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.846462 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:00:09.846657 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.846462 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:00:09.853679 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.853411 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:09.961513 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:09.961474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd\" (UID: \"5cc2ace6-e225-4a79-b38b-b4a288317227\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:10.062087 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.062048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd\" (UID: \"5cc2ace6-e225-4a79-b38b-b4a288317227\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:10.062721 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.062701 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd\" (UID: \"5cc2ace6-e225-4a79-b38b-b4a288317227\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:10.159298 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.159211 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:10.295371 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.295343 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:10.297767 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:00:10.297740 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc2ace6_e225_4a79_b38b_b4a288317227.slice/crio-906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb WatchSource:0}: Error finding container 906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb: Status 404 returned error can't find the container with id 906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb Jul 16 21:00:10.886649 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.886537 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerStarted","Data":"96a6f1a6c04e6b5195367230cc221c37b5672becd46d6746ba539e0c6533014e"} Jul 16 21:00:10.886649 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.886619 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerStarted","Data":"906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb"} Jul 16 21:00:10.902546 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:10.902478 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" podStartSLOduration=1.90246037 podStartE2EDuration="1.90246037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:00:10.900795011 +0000 UTC m=+1618.742646139" watchObservedRunningTime="2026-07-16 21:00:10.90246037 +0000 UTC m=+1618.744311508" Jul 16 21:00:11.894195 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:11.894162 2582 generic.go:356] "Generic (PLEG): container finished" podID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerID="96a6f1a6c04e6b5195367230cc221c37b5672becd46d6746ba539e0c6533014e" exitCode=7 Jul 16 21:00:11.894606 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:11.894210 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerDied","Data":"96a6f1a6c04e6b5195367230cc221c37b5672becd46d6746ba539e0c6533014e"} Jul 16 21:00:11.894606 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:11.894440 2582 scope.go:117] "RemoveContainer" containerID="96a6f1a6c04e6b5195367230cc221c37b5672becd46d6746ba539e0c6533014e" Jul 16 21:00:12.632271 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:12.632228 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:00:12.900886 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:12.900846 2582 generic.go:356] "Generic (PLEG): container finished" podID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerID="86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954" exitCode=7 Jul 16 21:00:12.901376 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:12.900922 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerDied","Data":"86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954"} Jul 16 21:00:12.901376 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:12.900988 2582 scope.go:117] "RemoveContainer" containerID="96a6f1a6c04e6b5195367230cc221c37b5672becd46d6746ba539e0c6533014e" Jul 16 21:00:12.901376 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:12.901236 2582 scope.go:117] "RemoveContainer" containerID="86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954" Jul 16 21:00:12.901537 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:00:12.901452 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd_odh-ai-gateway-infra(5cc2ace6-e225-4a79-b38b-b4a288317227)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" Jul 16 21:00:13.908565 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:13.908533 2582 scope.go:117] "RemoveContainer" containerID="86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954" Jul 16 21:00:13.908945 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:00:13.908757 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd_odh-ai-gateway-infra(5cc2ace6-e225-4a79-b38b-b4a288317227)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" Jul 16 21:00:24.722529 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:24.722502 2582 scope.go:117] "RemoveContainer" containerID="86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954" Jul 16 21:00:25.973638 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:25.973606 2582 generic.go:356] "Generic (PLEG): container finished" podID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerID="e81f8208346b7a535dc3b3c0a0049faba32de677c13affdcede652f1fa1ca846" exitCode=0 Jul 16 21:00:25.974095 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:25.973681 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerDied","Data":"e81f8208346b7a535dc3b3c0a0049faba32de677c13affdcede652f1fa1ca846"} Jul 16 21:00:25.974095 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:25.973765 2582 scope.go:117] "RemoveContainer" containerID="86ea77b7f94fa4191a06a397f43ca677758e76a7022b7dfe48abf0fa2eaf2954" Jul 16 21:00:27.002684 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.002652 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:27.119318 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.119291 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:27.217461 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.217428 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca\") pod \"5cc2ace6-e225-4a79-b38b-b4a288317227\" (UID: \"5cc2ace6-e225-4a79-b38b-b4a288317227\") " Jul 16 21:00:27.217847 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.217825 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5cc2ace6-e225-4a79-b38b-b4a288317227" (UID: "5cc2ace6-e225-4a79-b38b-b4a288317227"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:00:27.319028 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.318993 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc2ace6-e225-4a79-b38b-b4a288317227-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:00:27.986152 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.986113 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" event={"ID":"5cc2ace6-e225-4a79-b38b-b4a288317227","Type":"ContainerDied","Data":"906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb"} Jul 16 21:00:27.986152 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.986143 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd" Jul 16 21:00:27.986370 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:27.986168 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="906d7fdc0874f24bfc3720fe1c5953f7def7cebfac72283177e7b20b0436adbb" Jul 16 21:00:28.017667 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:28.017627 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:28.021972 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:28.021944 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-mbxmd"] Jul 16 21:00:28.723900 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:28.723860 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" path="/var/lib/kubelet/pods/5cc2ace6-e225-4a79-b38b-b4a288317227/volumes" Jul 16 21:00:59.885628 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.885590 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5"] Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.885965 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.885976 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.885991 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.885997 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.886004 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.886010 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.886063 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.886278 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.886071 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:00:59.889150 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.889130 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:00:59.891458 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.891437 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:00:59.891591 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.891457 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:00:59.896140 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.896103 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5"] Jul 16 21:00:59.905841 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:00:59.905618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-nccd5\" (UID: \"09520c6b-bde6-4660-bac9-937fd8e8fd23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:00.006738 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:00.006701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-nccd5\" (UID: \"09520c6b-bde6-4660-bac9-937fd8e8fd23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:00.007330 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:00.007310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18adf534-nccd5\" (UID: \"09520c6b-bde6-4660-bac9-937fd8e8fd23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:00.200743 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:00.200651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:00.331371 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:00.331342 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5"] Jul 16 21:01:00.333461 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:01:00.333426 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09520c6b_bde6_4660_bac9_937fd8e8fd23.slice/crio-310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0 WatchSource:0}: Error finding container 310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0: Status 404 returned error can't find the container with id 310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0 Jul 16 21:01:01.176282 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:01.176245 2582 generic.go:356] "Generic (PLEG): container finished" podID="09520c6b-bde6-4660-bac9-937fd8e8fd23" containerID="937a93076708631722297c97cb1e76742827f2a9bc9c69ce51c92239c01b31cf" exitCode=0 Jul 16 21:01:01.176710 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:01.176302 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" event={"ID":"09520c6b-bde6-4660-bac9-937fd8e8fd23","Type":"ContainerDied","Data":"937a93076708631722297c97cb1e76742827f2a9bc9c69ce51c92239c01b31cf"} Jul 16 21:01:01.176710 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:01.176342 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" event={"ID":"09520c6b-bde6-4660-bac9-937fd8e8fd23","Type":"ContainerStarted","Data":"310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0"} Jul 16 21:01:02.315339 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:02.315311 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:02.326628 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:02.326597 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca\") pod \"09520c6b-bde6-4660-bac9-937fd8e8fd23\" (UID: \"09520c6b-bde6-4660-bac9-937fd8e8fd23\") " Jul 16 21:01:02.326936 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:02.326914 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09520c6b-bde6-4660-bac9-937fd8e8fd23" (UID: "09520c6b-bde6-4660-bac9-937fd8e8fd23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:01:02.427363 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:02.427325 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09520c6b-bde6-4660-bac9-937fd8e8fd23-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:01:03.188068 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:03.188035 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" Jul 16 21:01:03.188254 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:03.188037 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18adf534-nccd5" event={"ID":"09520c6b-bde6-4660-bac9-937fd8e8fd23","Type":"ContainerDied","Data":"310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0"} Jul 16 21:01:03.188254 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:03.188169 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="310c10040e4e1303a18674b946929172ad8974cdf45276689ebcc1c35b19b4f0" Jul 16 21:01:12.932360 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:12.932320 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:01:21.334798 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:21.334748 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:01:22.570079 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.570041 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:22.570492 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.570383 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09520c6b-bde6-4660-bac9-937fd8e8fd23" containerName="revoke-keys" Jul 16 21:01:22.570492 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.570393 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09520c6b-bde6-4660-bac9-937fd8e8fd23" containerName="revoke-keys" Jul 16 21:01:22.570492 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.570466 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09520c6b-bde6-4660-bac9-937fd8e8fd23" containerName="revoke-keys" Jul 16 21:01:22.570492 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.570474 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cc2ace6-e225-4a79-b38b-b4a288317227" containerName="revoke-keys" Jul 16 21:01:22.573515 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.573498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:22.575809 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.575787 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:01:22.575927 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.575819 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:01:22.580739 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.580710 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:22.704913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.704872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-fgd67\" (UID: \"396e6343-fc31-477b-a759-5a6ba3a2f17b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:22.805745 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.805692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-fgd67\" (UID: \"396e6343-fc31-477b-a759-5a6ba3a2f17b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:22.806329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.806304 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-fgd67\" (UID: \"396e6343-fc31-477b-a759-5a6ba3a2f17b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:22.885090 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:22.885006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:23.024988 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:23.024955 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:23.027204 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:01:23.027177 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396e6343_fc31_477b_a759_5a6ba3a2f17b.slice/crio-7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b WatchSource:0}: Error finding container 7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b: Status 404 returned error can't find the container with id 7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b Jul 16 21:01:23.296995 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:23.296955 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerStarted","Data":"7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b"} Jul 16 21:01:24.304290 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:24.304257 2582 generic.go:356] "Generic (PLEG): container finished" podID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerID="400f14c26720fa33dc10324eb1cc839bb61724a833f8504220dca0872070c52a" exitCode=7 Jul 16 21:01:24.304687 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:24.304316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerDied","Data":"400f14c26720fa33dc10324eb1cc839bb61724a833f8504220dca0872070c52a"} Jul 16 21:01:24.304687 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:24.304504 2582 scope.go:117] "RemoveContainer" containerID="400f14c26720fa33dc10324eb1cc839bb61724a833f8504220dca0872070c52a" Jul 16 21:01:25.311065 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:25.311033 2582 generic.go:356] "Generic (PLEG): container finished" podID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerID="9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08" exitCode=7 Jul 16 21:01:25.311504 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:25.311127 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerDied","Data":"9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08"} Jul 16 21:01:25.311504 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:25.311186 2582 scope.go:117] "RemoveContainer" containerID="400f14c26720fa33dc10324eb1cc839bb61724a833f8504220dca0872070c52a" Jul 16 21:01:25.311504 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:25.311319 2582 scope.go:117] "RemoveContainer" containerID="9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08" Jul 16 21:01:25.311653 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:01:25.311544 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-25df27a3-fgd67_odh-ai-gateway-infra(396e6343-fc31-477b-a759-5a6ba3a2f17b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" Jul 16 21:01:26.318939 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:26.318910 2582 scope.go:117] "RemoveContainer" containerID="9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08" Jul 16 21:01:26.319341 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:01:26.319090 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-25df27a3-fgd67_odh-ai-gateway-infra(396e6343-fc31-477b-a759-5a6ba3a2f17b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" Jul 16 21:01:41.717451 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:41.717375 2582 scope.go:117] "RemoveContainer" containerID="9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08" Jul 16 21:01:42.409278 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:42.409232 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerStarted","Data":"d6e2b3564428254e77df3cd98f023c72f56b91496212f218d6c73681744b7b86"} Jul 16 21:01:42.426123 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:42.426068 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" podStartSLOduration=20.426048814 podStartE2EDuration="20.426048814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:01:42.423265261 +0000 UTC m=+1710.265116380" watchObservedRunningTime="2026-07-16 21:01:42.426048814 +0000 UTC m=+1710.267899941" Jul 16 21:01:43.416282 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:43.416239 2582 generic.go:356] "Generic (PLEG): container finished" podID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerID="d6e2b3564428254e77df3cd98f023c72f56b91496212f218d6c73681744b7b86" exitCode=0 Jul 16 21:01:43.416847 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:43.416309 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerDied","Data":"d6e2b3564428254e77df3cd98f023c72f56b91496212f218d6c73681744b7b86"} Jul 16 21:01:43.416847 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:43.416372 2582 scope.go:117] "RemoveContainer" containerID="9c3fa0bf92d25243ff30fd87861841eaa33414ea5617ecccbe5ef659b862fa08" Jul 16 21:01:43.433293 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:43.433259 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:44.584262 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:44.584221 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:44.709038 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:44.708939 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca\") pod \"396e6343-fc31-477b-a759-5a6ba3a2f17b\" (UID: \"396e6343-fc31-477b-a759-5a6ba3a2f17b\") " Jul 16 21:01:44.709330 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:44.709305 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "396e6343-fc31-477b-a759-5a6ba3a2f17b" (UID: "396e6343-fc31-477b-a759-5a6ba3a2f17b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:01:44.810395 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:44.810351 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/396e6343-fc31-477b-a759-5a6ba3a2f17b-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:01:45.429312 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:45.429275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" event={"ID":"396e6343-fc31-477b-a759-5a6ba3a2f17b","Type":"ContainerDied","Data":"7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b"} Jul 16 21:01:45.429312 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:45.429307 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67" Jul 16 21:01:45.429531 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:45.429324 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f7ca134e62676d1c61a7c4966b9daf72ee107168249da0cb52f2fad62ff3f8b" Jul 16 21:01:45.456020 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:45.455961 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:45.458638 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:45.458612 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-fgd67"] Jul 16 21:01:46.723332 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:01:46.723298 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" path="/var/lib/kubelet/pods/396e6343-fc31-477b-a759-5a6ba3a2f17b/volumes" Jul 16 21:02:22.613199 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613162 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7"] Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613506 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613519 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613544 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613550 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613614 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.613868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.613621 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:22.616654 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.616636 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:22.619367 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.619345 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:02:22.619495 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.619426 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:02:22.624737 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.624490 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7"] Jul 16 21:02:22.744310 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.744281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7\" (UID: \"d9d8c42e-8b04-4861-9160-768538ba7a20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:22.845050 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.845013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7\" (UID: \"d9d8c42e-8b04-4861-9160-768538ba7a20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:22.845701 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.845677 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7\" (UID: \"d9d8c42e-8b04-4861-9160-768538ba7a20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:22.929312 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:22.929210 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:23.073239 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:23.073210 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7"] Jul 16 21:02:23.075152 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:02:23.075122 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d8c42e_8b04_4861_9160_768538ba7a20.slice/crio-8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940 WatchSource:0}: Error finding container 8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940: Status 404 returned error can't find the container with id 8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940 Jul 16 21:02:23.638638 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:23.638600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" event={"ID":"d9d8c42e-8b04-4861-9160-768538ba7a20","Type":"ContainerStarted","Data":"fc46c0966e831686fff2848b5901420f9556693783f78a7dd8702abc5a502582"} Jul 16 21:02:23.639167 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:23.638657 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" event={"ID":"d9d8c42e-8b04-4861-9160-768538ba7a20","Type":"ContainerStarted","Data":"8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940"} Jul 16 21:02:23.654156 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:23.654100 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" podStartSLOduration=1.654084778 podStartE2EDuration="1.654084778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:02:23.653822495 +0000 UTC m=+1751.495673622" watchObservedRunningTime="2026-07-16 21:02:23.654084778 +0000 UTC m=+1751.495935904" Jul 16 21:02:24.645482 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:24.645439 2582 generic.go:356] "Generic (PLEG): container finished" podID="d9d8c42e-8b04-4861-9160-768538ba7a20" containerID="fc46c0966e831686fff2848b5901420f9556693783f78a7dd8702abc5a502582" exitCode=0 Jul 16 21:02:24.645891 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:24.645528 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" event={"ID":"d9d8c42e-8b04-4861-9160-768538ba7a20","Type":"ContainerDied","Data":"fc46c0966e831686fff2848b5901420f9556693783f78a7dd8702abc5a502582"} Jul 16 21:02:25.790530 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:25.790505 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:25.869086 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:25.869050 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca\") pod \"d9d8c42e-8b04-4861-9160-768538ba7a20\" (UID: \"d9d8c42e-8b04-4861-9160-768538ba7a20\") " Jul 16 21:02:25.869401 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:25.869378 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9d8c42e-8b04-4861-9160-768538ba7a20" (UID: "d9d8c42e-8b04-4861-9160-768538ba7a20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:02:25.970632 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:25.970523 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d8c42e-8b04-4861-9160-768538ba7a20-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:02:26.657698 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:26.657669 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" Jul 16 21:02:26.657866 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:26.657698 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7" event={"ID":"d9d8c42e-8b04-4861-9160-768538ba7a20","Type":"ContainerDied","Data":"8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940"} Jul 16 21:02:26.657866 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:26.657739 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a5de170e6ec81cabb8b65f51ecd7965e82d0ed73fd58156d30fbd9f2fdcc940" Jul 16 21:02:35.857944 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:35.857902 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:02:49.136871 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:49.136828 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:02:50.883176 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883137 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:02:50.883603 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883489 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d8c42e-8b04-4861-9160-768538ba7a20" containerName="revoke-keys" Jul 16 21:02:50.883603 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883500 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d8c42e-8b04-4861-9160-768538ba7a20" containerName="revoke-keys" Jul 16 21:02:50.883603 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883519 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:50.883603 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883527 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:50.883752 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883640 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d8c42e-8b04-4861-9160-768538ba7a20" containerName="revoke-keys" Jul 16 21:02:50.883752 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.883650 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="396e6343-fc31-477b-a759-5a6ba3a2f17b" containerName="revoke-keys" Jul 16 21:02:50.886752 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.886734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:02:50.889498 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.889475 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:02:50.889626 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.889552 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:02:50.895960 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.895934 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:02:50.989731 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:50.989689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p\" (UID: \"dad2f768-58b3-4f2b-89bc-34314add83e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:02:51.090829 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.090773 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p\" (UID: \"dad2f768-58b3-4f2b-89bc-34314add83e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:02:51.091404 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.091384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p\" (UID: \"dad2f768-58b3-4f2b-89bc-34314add83e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:02:51.198540 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.198447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:02:51.343315 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.343283 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:02:51.343908 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:02:51.343873 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad2f768_58b3_4f2b_89bc_34314add83e8.slice/crio-f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781 WatchSource:0}: Error finding container f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781: Status 404 returned error can't find the container with id f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781 Jul 16 21:02:51.802689 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.802595 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerStarted","Data":"4c4afc3aca4325b6e0245f989eafd9c60bd269d31a069767a56eb19b9aa48251"} Jul 16 21:02:51.802689 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.802647 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerStarted","Data":"f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781"} Jul 16 21:02:51.818649 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:51.818558 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" podStartSLOduration=1.818540068 podStartE2EDuration="1.818540068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:02:51.817423837 +0000 UTC m=+1779.659274963" watchObservedRunningTime="2026-07-16 21:02:51.818540068 +0000 UTC m=+1779.660391194" Jul 16 21:02:52.809712 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:52.809675 2582 generic.go:356] "Generic (PLEG): container finished" podID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerID="4c4afc3aca4325b6e0245f989eafd9c60bd269d31a069767a56eb19b9aa48251" exitCode=7 Jul 16 21:02:52.810155 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:52.809759 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerDied","Data":"4c4afc3aca4325b6e0245f989eafd9c60bd269d31a069767a56eb19b9aa48251"} Jul 16 21:02:52.810155 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:52.809922 2582 scope.go:117] "RemoveContainer" containerID="4c4afc3aca4325b6e0245f989eafd9c60bd269d31a069767a56eb19b9aa48251" Jul 16 21:02:53.817449 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:53.817414 2582 generic.go:356] "Generic (PLEG): container finished" podID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerID="4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5" exitCode=7 Jul 16 21:02:53.817882 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:53.817497 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerDied","Data":"4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5"} Jul 16 21:02:53.817882 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:53.817557 2582 scope.go:117] "RemoveContainer" containerID="4c4afc3aca4325b6e0245f989eafd9c60bd269d31a069767a56eb19b9aa48251" Jul 16 21:02:53.817882 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:53.817715 2582 scope.go:117] "RemoveContainer" containerID="4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5" Jul 16 21:02:53.818012 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:02:53.817942 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p_odh-ai-gateway-infra(dad2f768-58b3-4f2b-89bc-34314add83e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" Jul 16 21:02:54.824776 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:02:54.824750 2582 scope.go:117] "RemoveContainer" containerID="4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5" Jul 16 21:02:54.825190 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:02:54.824970 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p_odh-ai-gateway-infra(dad2f768-58b3-4f2b-89bc-34314add83e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" Jul 16 21:03:09.717300 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:09.717199 2582 scope.go:117] "RemoveContainer" containerID="4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5" Jul 16 21:03:10.914835 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:10.914800 2582 generic.go:356] "Generic (PLEG): container finished" podID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerID="1d26349407b0b7ae202eeb8aaf60604899d6d6c17c07ce24a2a643fc84392f74" exitCode=0 Jul 16 21:03:10.915253 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:10.914874 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerDied","Data":"1d26349407b0b7ae202eeb8aaf60604899d6d6c17c07ce24a2a643fc84392f74"} Jul 16 21:03:10.915253 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:10.914933 2582 scope.go:117] "RemoveContainer" containerID="4ebf28ccc0d5a1e60791dd860cdb4bfe5ab4b105a3b54a9515462bb508ba5ed5" Jul 16 21:03:11.946722 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:11.946689 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:03:12.066844 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.066816 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:03:12.183330 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.183246 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca\") pod \"dad2f768-58b3-4f2b-89bc-34314add83e8\" (UID: \"dad2f768-58b3-4f2b-89bc-34314add83e8\") " Jul 16 21:03:12.183624 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.183602 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dad2f768-58b3-4f2b-89bc-34314add83e8" (UID: "dad2f768-58b3-4f2b-89bc-34314add83e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:03:12.284004 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.283962 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dad2f768-58b3-4f2b-89bc-34314add83e8-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:03:12.777340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.777310 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:03:12.781225 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.781204 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:03:12.791417 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.791397 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:03:12.795598 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.795561 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:03:12.928637 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.928603 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" Jul 16 21:03:12.928809 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.928607 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p" event={"ID":"dad2f768-58b3-4f2b-89bc-34314add83e8","Type":"ContainerDied","Data":"f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781"} Jul 16 21:03:12.928809 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.928732 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6fd63014ea8c8b49b945d3fbef4166f94c0b5fdfafeb9e7067dd0384d2fe781" Jul 16 21:03:12.960316 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.960285 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:03:12.964506 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:12.964477 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-rdh4p"] Jul 16 21:03:14.723011 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:14.722978 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" path="/var/lib/kubelet/pods/dad2f768-58b3-4f2b-89bc-34314add83e8/volumes" Jul 16 21:03:50.956275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.956233 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6"] Jul 16 21:03:50.963061 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963020 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.963061 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963056 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.963270 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963087 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.963270 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963093 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.963338 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963289 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.963338 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.963298 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:03:50.966207 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.966188 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:50.969735 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.969709 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:03:50.969849 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.969780 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:03:50.976193 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:50.976160 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6"] Jul 16 21:03:51.022549 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:51.022510 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-242l6\" (UID: \"fa01a427-576c-4d71-abc2-2e5950cb999c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:51.123475 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:51.123432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-242l6\" (UID: \"fa01a427-576c-4d71-abc2-2e5950cb999c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:51.124099 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:51.124077 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f05f95fc-242l6\" (UID: \"fa01a427-576c-4d71-abc2-2e5950cb999c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:51.278078 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:51.278001 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:51.418680 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:51.418647 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6"] Jul 16 21:03:51.422026 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:03:51.421998 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa01a427_576c_4d71_abc2_2e5950cb999c.slice/crio-ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a WatchSource:0}: Error finding container ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a: Status 404 returned error can't find the container with id ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a Jul 16 21:03:52.154962 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:52.154923 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa01a427-576c-4d71-abc2-2e5950cb999c" containerID="229142964bd7e3bdc78bede7cfffb4c5b4fb1f605f68a5fbfbb7aa800f0c685c" exitCode=0 Jul 16 21:03:52.155354 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:52.155012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" event={"ID":"fa01a427-576c-4d71-abc2-2e5950cb999c","Type":"ContainerDied","Data":"229142964bd7e3bdc78bede7cfffb4c5b4fb1f605f68a5fbfbb7aa800f0c685c"} Jul 16 21:03:52.155354 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:52.155068 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" event={"ID":"fa01a427-576c-4d71-abc2-2e5950cb999c","Type":"ContainerStarted","Data":"ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a"} Jul 16 21:03:53.291748 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:53.291723 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:53.344495 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:53.344467 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca\") pod \"fa01a427-576c-4d71-abc2-2e5950cb999c\" (UID: \"fa01a427-576c-4d71-abc2-2e5950cb999c\") " Jul 16 21:03:53.344819 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:53.344798 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa01a427-576c-4d71-abc2-2e5950cb999c" (UID: "fa01a427-576c-4d71-abc2-2e5950cb999c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:03:53.445403 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:53.445308 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa01a427-576c-4d71-abc2-2e5950cb999c-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:03:54.166772 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:54.166743 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" Jul 16 21:03:54.166772 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:54.166749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f05f95fc-242l6" event={"ID":"fa01a427-576c-4d71-abc2-2e5950cb999c","Type":"ContainerDied","Data":"ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a"} Jul 16 21:03:54.167000 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:03:54.166791 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee009052e8883ef3aa7954d53294a072d979a82552b6a78d916cbdce98c6ed9a" Jul 16 21:04:03.751347 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:04:03.751298 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:05:12.876180 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:05:12.876144 2582 scope.go:117] "RemoveContainer" containerID="94dadd376a62d8d759df4c7b6632fe61a59eea09d5b0dbd89c2076f8174b4f3b" Jul 16 21:05:42.328022 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:05:42.327986 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:05:53.338031 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:05:53.337997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:06:02.934534 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:02.934492 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:06:09.656684 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.656646 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh"] Jul 16 21:06:09.657138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657035 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa01a427-576c-4d71-abc2-2e5950cb999c" containerName="revoke-keys" Jul 16 21:06:09.657138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657048 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa01a427-576c-4d71-abc2-2e5950cb999c" containerName="revoke-keys" Jul 16 21:06:09.657138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657056 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:06:09.657138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657062 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:06:09.657138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657136 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa01a427-576c-4d71-abc2-2e5950cb999c" containerName="revoke-keys" Jul 16 21:06:09.657321 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.657146 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dad2f768-58b3-4f2b-89bc-34314add83e8" containerName="revoke-keys" Jul 16 21:06:09.660310 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.660292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:09.662938 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.662910 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:06:09.663086 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.663001 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:06:09.669326 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.669291 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh"] Jul 16 21:06:09.761805 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.761755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh\" (UID: \"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:09.862382 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.862339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh\" (UID: \"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:09.863147 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.863127 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh\" (UID: \"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:09.972762 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:09.972663 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:10.114644 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:10.114617 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh"] Jul 16 21:06:10.116783 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:06:10.116754 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0583efc8_8ab2_4a0e_8cdf_5bf731e87b57.slice/crio-1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67 WatchSource:0}: Error finding container 1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67: Status 404 returned error can't find the container with id 1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67 Jul 16 21:06:10.934124 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:10.934084 2582 generic.go:356] "Generic (PLEG): container finished" podID="0583efc8-8ab2-4a0e-8cdf-5bf731e87b57" containerID="55bc307eb20bc9cf319450041ac263576826cc579f188109ff4e739be665efea" exitCode=0 Jul 16 21:06:10.934526 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:10.934135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" event={"ID":"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57","Type":"ContainerDied","Data":"55bc307eb20bc9cf319450041ac263576826cc579f188109ff4e739be665efea"} Jul 16 21:06:10.934526 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:10.934182 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" event={"ID":"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57","Type":"ContainerStarted","Data":"1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67"} Jul 16 21:06:12.077815 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.077782 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:12.185131 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.185098 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca\") pod \"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57\" (UID: \"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57\") " Jul 16 21:06:12.185522 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.185495 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0583efc8-8ab2-4a0e-8cdf-5bf731e87b57" (UID: "0583efc8-8ab2-4a0e-8cdf-5bf731e87b57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:06:12.286524 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.286490 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0583efc8-8ab2-4a0e-8cdf-5bf731e87b57-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:06:12.946664 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.946634 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" Jul 16 21:06:12.946845 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.946668 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh" event={"ID":"0583efc8-8ab2-4a0e-8cdf-5bf731e87b57","Type":"ContainerDied","Data":"1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67"} Jul 16 21:06:12.946845 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:12.946723 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e98ae083fc4c26166c0ca6b91b664c9cea63e4fd926268f12d50f4a4b90eb67" Jul 16 21:06:22.728408 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:22.728360 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:06:59.237766 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:06:59.237731 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:07:10.047965 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:10.047910 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:07:12.906540 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:12.906509 2582 scope.go:117] "RemoveContainer" containerID="e81f8208346b7a535dc3b3c0a0049faba32de677c13affdcede652f1fa1ca846" Jul 16 21:07:23.751651 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:23.751615 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:07:26.979725 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.979690 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b"] Jul 16 21:07:26.980131 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.980044 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0583efc8-8ab2-4a0e-8cdf-5bf731e87b57" containerName="revoke-keys" Jul 16 21:07:26.980131 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.980055 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0583efc8-8ab2-4a0e-8cdf-5bf731e87b57" containerName="revoke-keys" Jul 16 21:07:26.980131 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.980128 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0583efc8-8ab2-4a0e-8cdf-5bf731e87b57" containerName="revoke-keys" Jul 16 21:07:26.983280 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.983259 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:26.985905 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.985874 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:07:26.986049 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.985938 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:07:26.990533 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:26.990507 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b"] Jul 16 21:07:27.054605 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:27.054533 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b\" (UID: \"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:27.155786 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:27.155741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b\" (UID: \"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:27.156451 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:27.156420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b\" (UID: \"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:27.295434 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:27.295394 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:27.441147 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:27.441113 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b"] Jul 16 21:07:27.442865 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:07:27.442834 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0f41dc_a3ef_4cd3_94ed_9616b5b4db25.slice/crio-a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370 WatchSource:0}: Error finding container a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370: Status 404 returned error can't find the container with id a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370 Jul 16 21:07:28.373101 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:28.373059 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25" containerID="24816be5b1ca449d2ae87c481959840df029a3e6a52de51a86f7a4051f8effeb" exitCode=0 Jul 16 21:07:28.373464 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:28.373145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" event={"ID":"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25","Type":"ContainerDied","Data":"24816be5b1ca449d2ae87c481959840df029a3e6a52de51a86f7a4051f8effeb"} Jul 16 21:07:28.373464 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:28.373228 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" event={"ID":"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25","Type":"ContainerStarted","Data":"a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370"} Jul 16 21:07:29.517355 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:29.517322 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:29.576977 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:29.576934 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca\") pod \"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25\" (UID: \"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25\") " Jul 16 21:07:29.577331 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:29.577306 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25" (UID: "6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:07:29.678465 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:29.678375 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:07:30.384975 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:30.384944 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" Jul 16 21:07:30.385226 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:30.384942 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b" event={"ID":"6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25","Type":"ContainerDied","Data":"a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370"} Jul 16 21:07:30.385226 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:30.385070 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a32c9cb25894de3998a8997ad1f8c6b57624414f2299eae1b015ff25b25d1370" Jul 16 21:07:40.751942 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:40.751851 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:07:49.240583 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:07:49.240540 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:08:12.820448 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:12.820416 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:08:12.824926 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:12.824895 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:08:12.835031 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:12.835008 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:08:12.839207 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:12.839188 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:08:12.934669 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:12.934636 2582 scope.go:117] "RemoveContainer" containerID="d6e2b3564428254e77df3cd98f023c72f56b91496212f218d6c73681744b7b86" Jul 16 21:08:42.703789 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:42.703750 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:08:55.648344 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:08:55.648304 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:09:12.956707 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:12.956672 2582 scope.go:117] "RemoveContainer" containerID="1d26349407b0b7ae202eeb8aaf60604899d6d6c17c07ce24a2a643fc84392f74" Jul 16 21:09:45.239979 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.239944 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:09:45.240459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.240319 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25" containerName="revoke-keys" Jul 16 21:09:45.240459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.240330 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25" containerName="revoke-keys" Jul 16 21:09:45.240459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.240398 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25" containerName="revoke-keys" Jul 16 21:09:45.244028 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.244008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.247081 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.247062 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-054dd3-openshift-default-dockercfg-rrzxs\"" Jul 16 21:09:45.264837 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.264806 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:09:45.386121 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386085 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmckq\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386121 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386126 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386308 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386542 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386419 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.386542 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.386456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487503 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487503 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487612 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmckq\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.487791 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.487704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.488340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.488096 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.488340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.488241 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.488340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.488285 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.488340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.488297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.488530 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.488490 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.490167 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.490106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.490378 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.490358 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.497002 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.496973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.497783 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.497747 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmckq\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq\") pod \"e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.554778 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.554737 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:45.708190 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.708161 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:09:45.709727 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:09:45.709702 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd27d05_a214_4741_a40a_9bcf400d52d8.slice/crio-b33cc4120f9114a08ccfbd6c5cd05d5609f63586b9d42b85dbb86d3c4ee6a4e0 WatchSource:0}: Error finding container b33cc4120f9114a08ccfbd6c5cd05d5609f63586b9d42b85dbb86d3c4ee6a4e0: Status 404 returned error can't find the container with id b33cc4120f9114a08ccfbd6c5cd05d5609f63586b9d42b85dbb86d3c4ee6a4e0 Jul 16 21:09:45.712346 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:45.712327 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:09:46.147800 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:46.147763 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerStarted","Data":"b33cc4120f9114a08ccfbd6c5cd05d5609f63586b9d42b85dbb86d3c4ee6a4e0"} Jul 16 21:09:47.951968 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:47.951928 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:09:48.897568 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:48.897525 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:09:48.897688 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:48.897624 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:09:48.897688 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:48.897664 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:09:49.169248 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:49.169144 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerStarted","Data":"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be"} Jul 16 21:09:49.191128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:49.191071 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podStartSLOduration=1.006397704 podStartE2EDuration="4.191052987s" totalImagesPullingTime="3.184655283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:09:49.190206081 +0000 UTC m=+2197.032057220" watchObservedRunningTime="2026-07-16 21:09:49.191052987 +0000 UTC m=+2197.032904114" Jul 16 21:09:49.555334 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:49.555293 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:49.560208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:49.560180 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:50.174168 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:50.174138 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:50.175126 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:50.175106 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:09:56.342086 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:09:56.342043 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:10:05.175950 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.175904 2582 patch_prober.go:28] interesting pod/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 16 21:10:05.176386 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.175976 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 16 21:10:05.276099 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.276000 2582 generic.go:356] "Generic (PLEG): container finished" podID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerID="cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be" exitCode=0 Jul 16 21:10:05.276099 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.276077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerDied","Data":"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be"} Jul 16 21:10:05.276565 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.276546 2582 scope.go:117] "RemoveContainer" containerID="cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be" Jul 16 21:10:05.278683 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.278648 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:10:05.278759 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.278726 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:10:05.278804 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:05.278761 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:10:06.284242 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:06.284203 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerStarted","Data":"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446"} Jul 16 21:10:06.555863 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:06.555764 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:10:06.556057 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:06.556032 2582 patch_prober.go:28] interesting pod/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 16 21:10:06.556102 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:06.556087 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 16 21:10:07.555952 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:07.555914 2582 patch_prober.go:28] interesting pod/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 16 21:10:07.556357 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:07.556009 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 16 21:10:08.555453 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:08.555406 2582 patch_prober.go:28] interesting pod/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 16 21:10:08.555680 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:08.555472 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 16 21:10:09.555797 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:09.555763 2582 patch_prober.go:28] interesting pod/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 16 21:10:09.556203 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:09.555818 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 16 21:10:10.559843 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:10.559812 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:10:10.560221 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:10.560150 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:10:10.560806 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:10.560785 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:10:12.936643 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:12.936605 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:10:27.944328 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:27.944285 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:10:41.327772 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:41.327680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:10:45.035646 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.035612 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4"] Jul 16 21:10:45.038174 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.038156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.042481 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:10:45.042443 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4\" is forbidden: User \"system:node:ip-10-0-135-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-240.ec2.internal' and this object" podUID="cd294a37-7a16-4cdb-be61-633ef44ad42b" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.042481 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.042475 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:10:45.042850 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.042835 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:10:45.055472 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.055437 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4"] Jul 16 21:10:45.201099 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.201056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4\" (UID: \"cd294a37-7a16-4cdb-be61-633ef44ad42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.302447 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.302349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4\" (UID: \"cd294a37-7a16-4cdb-be61-633ef44ad42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.303039 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.303018 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4\" (UID: \"cd294a37-7a16-4cdb-be61-633ef44ad42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.349066 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.349029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:45.483539 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.483510 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4"] Jul 16 21:10:45.484650 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:10:45.484621 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd294a37_7a16_4cdb_be61_633ef44ad42b.slice/crio-1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b WatchSource:0}: Error finding container 1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b: Status 404 returned error can't find the container with id 1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b Jul 16 21:10:45.501850 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:45.501822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" event={"ID":"cd294a37-7a16-4cdb-be61-633ef44ad42b","Type":"ContainerStarted","Data":"1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b"} Jul 16 21:10:46.509179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:46.509086 2582 generic.go:356] "Generic (PLEG): container finished" podID="cd294a37-7a16-4cdb-be61-633ef44ad42b" containerID="6ea8ca6cf8e30b0e61a7c77b418cffcc5f56107aecdea07d5b52b49a40e46fe4" exitCode=0 Jul 16 21:10:46.509179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:46.509126 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" event={"ID":"cd294a37-7a16-4cdb-be61-633ef44ad42b","Type":"ContainerDied","Data":"6ea8ca6cf8e30b0e61a7c77b418cffcc5f56107aecdea07d5b52b49a40e46fe4"} Jul 16 21:10:47.641565 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:47.641541 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:47.824407 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:47.824357 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca\") pod \"cd294a37-7a16-4cdb-be61-633ef44ad42b\" (UID: \"cd294a37-7a16-4cdb-be61-633ef44ad42b\") " Jul 16 21:10:47.824775 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:47.824753 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd294a37-7a16-4cdb-be61-633ef44ad42b" (UID: "cd294a37-7a16-4cdb-be61-633ef44ad42b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:10:47.926013 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:47.925984 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd294a37-7a16-4cdb-be61-633ef44ad42b-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:10:48.521495 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:48.521460 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" Jul 16 21:10:48.521495 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:48.521492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4" event={"ID":"cd294a37-7a16-4cdb-be61-633ef44ad42b","Type":"ContainerDied","Data":"1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b"} Jul 16 21:10:48.521738 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:48.521538 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1621b60fa99b283b4b08f0d95ffa61885071fe07c6dd8dd1348a50a92d1d121b" Jul 16 21:10:53.334367 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:10:53.334336 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:11:04.133776 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:04.133737 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:11:07.389263 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.389219 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs"] Jul 16 21:11:07.389667 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.389620 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd294a37-7a16-4cdb-be61-633ef44ad42b" containerName="revoke-keys" Jul 16 21:11:07.389667 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.389633 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd294a37-7a16-4cdb-be61-633ef44ad42b" containerName="revoke-keys" Jul 16 21:11:07.389740 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.389702 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd294a37-7a16-4cdb-be61-633ef44ad42b" containerName="revoke-keys" Jul 16 21:11:07.392198 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.392179 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:07.394518 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.394496 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:11:07.394672 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.394540 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:11:07.400313 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.400285 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs"] Jul 16 21:11:07.499318 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.499272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs\" (UID: \"4c927d5e-3674-4449-8d7f-323a52ca753d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:07.600344 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.600288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs\" (UID: \"4c927d5e-3674-4449-8d7f-323a52ca753d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:07.600955 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.600934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs\" (UID: \"4c927d5e-3674-4449-8d7f-323a52ca753d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:07.704286 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.704198 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:07.839669 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:07.839645 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs"] Jul 16 21:11:07.841332 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:11:07.841305 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c927d5e_3674_4449_8d7f_323a52ca753d.slice/crio-5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596 WatchSource:0}: Error finding container 5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596: Status 404 returned error can't find the container with id 5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596 Jul 16 21:11:08.633559 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:08.633525 2582 generic.go:356] "Generic (PLEG): container finished" podID="4c927d5e-3674-4449-8d7f-323a52ca753d" containerID="ca4434dd653b2f4a896550ef618e05466109baabd347bc4b37a510851cab32c1" exitCode=0 Jul 16 21:11:08.633929 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:08.633602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" event={"ID":"4c927d5e-3674-4449-8d7f-323a52ca753d","Type":"ContainerDied","Data":"ca4434dd653b2f4a896550ef618e05466109baabd347bc4b37a510851cab32c1"} Jul 16 21:11:08.633929 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:08.633655 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" event={"ID":"4c927d5e-3674-4449-8d7f-323a52ca753d","Type":"ContainerStarted","Data":"5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596"} Jul 16 21:11:09.768655 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:09.768622 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:09.824738 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:09.824703 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca\") pod \"4c927d5e-3674-4449-8d7f-323a52ca753d\" (UID: \"4c927d5e-3674-4449-8d7f-323a52ca753d\") " Jul 16 21:11:09.825093 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:09.825071 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c927d5e-3674-4449-8d7f-323a52ca753d" (UID: "4c927d5e-3674-4449-8d7f-323a52ca753d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:11:09.925718 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:09.925633 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c927d5e-3674-4449-8d7f-323a52ca753d-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:10.646036 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:10.646007 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" Jul 16 21:11:10.646036 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:10.646024 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs" event={"ID":"4c927d5e-3674-4449-8d7f-323a52ca753d","Type":"ContainerDied","Data":"5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596"} Jul 16 21:11:10.646262 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:10.646074 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b87e862c8221782b197581f4f8ac470da3482e38913ad7bd02f98cd047c6596" Jul 16 21:11:20.747470 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:20.747426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:11:29.252459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:29.252426 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:11:29.252999 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:29.252693 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" containerID="cri-o://25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446" gracePeriod=30 Jul 16 21:11:30.195560 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.195526 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:11:30.195912 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.195899 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c927d5e-3674-4449-8d7f-323a52ca753d" containerName="revoke-keys" Jul 16 21:11:30.195983 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.195913 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c927d5e-3674-4449-8d7f-323a52ca753d" containerName="revoke-keys" Jul 16 21:11:30.195983 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.195982 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c927d5e-3674-4449-8d7f-323a52ca753d" containerName="revoke-keys" Jul 16 21:11:30.199115 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.199098 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.201607 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.201566 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-99f81a-openshift-default-dockercfg-62x6n\"" Jul 16 21:11:30.217513 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.217490 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:11:30.310275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310322 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310388 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhldc\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.310782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.310500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.411794 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.411756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.411794 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.411805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412019 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.411926 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412019 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.411962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412103 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412018 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412103 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412035 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhldc\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412103 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412103 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412306 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412306 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412306 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412285 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412476 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412563 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.412746 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.412614 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.414535 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.414512 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.414702 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.414682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.422447 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.422426 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhldc\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.422544 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.422456 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token\") pod \"e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.509646 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.509541 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:30.655116 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.655090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:11:30.656383 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:11:30.656349 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d60bdc4_a123_48c5_9291_71631a497b1d.slice/crio-e6cd131f967dbcb4e6162c2a8b107013805b4b27ef84fedcec31dd281f53d121 WatchSource:0}: Error finding container e6cd131f967dbcb4e6162c2a8b107013805b4b27ef84fedcec31dd281f53d121: Status 404 returned error can't find the container with id e6cd131f967dbcb4e6162c2a8b107013805b4b27ef84fedcec31dd281f53d121 Jul 16 21:11:30.658837 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.658803 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:30.658963 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.658874 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:30.658963 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.658905 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:30.762032 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.761938 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerStarted","Data":"50e46d48298f5be985011686b8853e89aa622d502e03f123784ce5d7946a7132"} Jul 16 21:11:30.762032 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.762003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerStarted","Data":"e6cd131f967dbcb4e6162c2a8b107013805b4b27ef84fedcec31dd281f53d121"} Jul 16 21:11:30.786479 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:30.786427 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podStartSLOduration=0.78641001 podStartE2EDuration="786.41001ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:11:30.783778641 +0000 UTC m=+2298.625629770" watchObservedRunningTime="2026-07-16 21:11:30.78641001 +0000 UTC m=+2298.628261136" Jul 16 21:11:31.510174 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:31.510134 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:31.515148 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:31.515126 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:31.767894 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:31.767815 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:31.768911 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:31.768893 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:34.524461 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.524392 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:11:34.649028 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.648993 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649039 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649062 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649089 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649133 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649178 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649221 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmckq\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649297 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket\") pod \"ddd27d05-a214-4741-a40a-9bcf400d52d8\" (UID: \"ddd27d05-a214-4741-a40a-9bcf400d52d8\") " Jul 16 21:11:34.649483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649356 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:11:34.649483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649444 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:11:34.649762 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649486 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:11:34.649832 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649812 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddd27d05-a214-4741-a40a-9bcf400d52d8-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.649886 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649839 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.649886 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.649855 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.650087 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.650067 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:11:34.650153 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.650092 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:11:34.651637 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.651610 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:11:34.651739 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.651664 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq" (OuterVolumeSpecName: "kube-api-access-bmckq") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "kube-api-access-bmckq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:11:34.651800 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.651758 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:11:34.651800 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.651770 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddd27d05-a214-4741-a40a-9bcf400d52d8" (UID: "ddd27d05-a214-4741-a40a-9bcf400d52d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 21:11:34.751339 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751297 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmckq\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-kube-api-access-bmckq\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.751339 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751334 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.751339 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751344 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.751613 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751356 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.751613 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751369 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.751613 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.751377 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddd27d05-a214-4741-a40a-9bcf400d52d8-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:34.787199 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.787171 2582 generic.go:356] "Generic (PLEG): container finished" podID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerID="25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446" exitCode=0 Jul 16 21:11:34.787406 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.787218 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" Jul 16 21:11:34.787406 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.787258 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerDied","Data":"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446"} Jul 16 21:11:34.787406 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.787312 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw" event={"ID":"ddd27d05-a214-4741-a40a-9bcf400d52d8","Type":"ContainerDied","Data":"b33cc4120f9114a08ccfbd6c5cd05d5609f63586b9d42b85dbb86d3c4ee6a4e0"} Jul 16 21:11:34.787406 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.787335 2582 scope.go:117] "RemoveContainer" containerID="25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446" Jul 16 21:11:34.797316 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.797295 2582 scope.go:117] "RemoveContainer" containerID="cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be" Jul 16 21:11:34.806614 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.806564 2582 scope.go:117] "RemoveContainer" containerID="25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446" Jul 16 21:11:34.806917 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.806882 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:11:34.806988 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:11:34.806919 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446\": container with ID starting with 25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446 not found: ID does not exist" containerID="25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446" Jul 16 21:11:34.806988 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.806946 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446"} err="failed to get container status \"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446\": rpc error: code = NotFound desc = could not find container \"25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446\": container with ID starting with 25cdab358bc594140b35a049bb05b56a9d6151ab97ff8357931ad7ab01967446 not found: ID does not exist" Jul 16 21:11:34.806988 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.806965 2582 scope.go:117] "RemoveContainer" containerID="cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be" Jul 16 21:11:34.807259 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:11:34.807240 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be\": container with ID starting with cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be not found: ID does not exist" containerID="cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be" Jul 16 21:11:34.807317 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.807264 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be"} err="failed to get container status \"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be\": rpc error: code = NotFound desc = could not find container \"cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be\": container with ID starting with cf4b71eb75f1b5a70b0d7a0818e4ab9f62b82fcf0ef1ff9a4a8b80983e2d61be not found: ID does not exist" Jul 16 21:11:34.811123 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:34.811098 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-054dd3-openshift-default-57f4989cb7-r57gw"] Jul 16 21:11:35.980118 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:35.980073 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:11:36.723099 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:36.723057 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" path="/var/lib/kubelet/pods/ddd27d05-a214-4741-a40a-9bcf400d52d8/volumes" Jul 16 21:11:38.038865 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.038812 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:11:38.039976 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.039948 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:38.040137 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.040125 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:38.040265 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.040253 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:38.040351 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.040341 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:38.040527 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.040515 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:38.053148 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.049896 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.061724 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.061062 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a896c7-openshift-default-dockercfg-tsprf\"" Jul 16 21:11:38.061724 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.061522 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:11:38.181857 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.181814 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182056 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.181903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182056 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.181933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182056 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.181957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.182040 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.182095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.182168 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182208 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.182203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.182363 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.182232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95x47\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283244 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95x47\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283781 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283781 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283781 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283649 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283781 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283715 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.283921 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.283792 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.284041 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.284022 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.284248 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.284231 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.285807 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.285781 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.285954 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.285938 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.291878 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.291819 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.291878 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.291843 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95x47\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47\") pod \"e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.376622 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.376544 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:38.549748 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.549656 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:11:38.550705 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:11:38.550675 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b89e77f_33ea_4ea9_9a7e_a87a16bd99ae.slice/crio-9099d8203eee7683a63c6c6d2fe35f52ee1e0cd402c37d4c05f49cbc5d898339 WatchSource:0}: Error finding container 9099d8203eee7683a63c6c6d2fe35f52ee1e0cd402c37d4c05f49cbc5d898339: Status 404 returned error can't find the container with id 9099d8203eee7683a63c6c6d2fe35f52ee1e0cd402c37d4c05f49cbc5d898339 Jul 16 21:11:38.553143 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.553110 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:38.553238 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.553173 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:38.553238 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.553202 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:38.815005 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.814901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerStarted","Data":"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c"} Jul 16 21:11:38.815267 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.815237 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerStarted","Data":"9099d8203eee7683a63c6c6d2fe35f52ee1e0cd402c37d4c05f49cbc5d898339"} Jul 16 21:11:38.838436 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:38.838369 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podStartSLOduration=0.838343384 podStartE2EDuration="838.343384ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:11:38.834684502 +0000 UTC m=+2306.676535629" watchObservedRunningTime="2026-07-16 21:11:38.838343384 +0000 UTC m=+2306.680194511" Jul 16 21:11:39.377722 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:39.377682 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:39.378991 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:39.378967 2582 patch_prober.go:28] interesting pod/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 21:11:39.379101 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:39.379029 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 21:11:40.381010 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:40.380978 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:40.833914 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:40.833876 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:40.843374 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:40.843125 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:44.860838 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.860801 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerID="50e46d48298f5be985011686b8853e89aa622d502e03f123784ce5d7946a7132" exitCode=0 Jul 16 21:11:44.861301 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.860850 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerDied","Data":"50e46d48298f5be985011686b8853e89aa622d502e03f123784ce5d7946a7132"} Jul 16 21:11:44.861301 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.861201 2582 scope.go:117] "RemoveContainer" containerID="50e46d48298f5be985011686b8853e89aa622d502e03f123784ce5d7946a7132" Jul 16 21:11:44.862210 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.862180 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:44.862323 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.862263 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:44.862323 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:44.862312 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:45.872741 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:45.872712 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerStarted","Data":"e51057e0956633227d893c648b8ff0c74dca26b200c4694caedb20613a43a9c3"} Jul 16 21:11:46.509951 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:46.509917 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:46.511374 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:46.511348 2582 patch_prober.go:28] interesting pod/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 21:11:46.511565 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:46.511543 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 21:11:46.768325 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:46.768173 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:47.509852 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:47.509819 2582 patch_prober.go:28] interesting pod/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 21:11:47.510240 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:47.509870 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 21:11:48.510420 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:48.510388 2582 patch_prober.go:28] interesting pod/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 21:11:48.510852 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:48.510663 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 21:11:49.510144 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:49.510112 2582 patch_prober.go:28] interesting pod/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 16 21:11:49.510304 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:49.510176 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 16 21:11:50.514744 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:50.514712 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:50.515897 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:50.515876 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:11:51.638202 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:51.638168 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:11:52.922156 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.922124 2582 generic.go:356] "Generic (PLEG): container finished" podID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerID="2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c" exitCode=0 Jul 16 21:11:52.922597 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.922196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerDied","Data":"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c"} Jul 16 21:11:52.922597 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.922557 2582 scope.go:117] "RemoveContainer" containerID="2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c" Jul 16 21:11:52.923548 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.923515 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:52.923640 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.923593 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:52.923640 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:52.923624 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:11:53.458789 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.458747 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr"] Jul 16 21:11:53.459438 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.459418 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd27d05-a214-4741-a40a-9bcf400d52d8" containerName="istio-proxy" Jul 16 21:11:53.461723 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.461705 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:53.464163 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.464145 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:11:53.464287 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.464199 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:11:53.469472 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.469426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr"] Jul 16 21:11:53.522582 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.522546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr\" (UID: \"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:53.623804 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.623764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr\" (UID: \"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:53.624375 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.624355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr\" (UID: \"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:53.775726 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.775614 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:53.953688 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.953284 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr"] Jul 16 21:11:53.954169 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:53.953705 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerStarted","Data":"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47"} Jul 16 21:11:53.954592 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:11:53.954538 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ff80f3_7854_4f7a_93c2_f7a0dc8a0563.slice/crio-09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01 WatchSource:0}: Error finding container 09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01: Status 404 returned error can't find the container with id 09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01 Jul 16 21:11:54.377439 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.377397 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:54.378523 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.378493 2582 patch_prober.go:28] interesting pod/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 21:11:54.378663 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.378545 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 21:11:54.960977 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.960944 2582 generic.go:356] "Generic (PLEG): container finished" podID="e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563" containerID="c399c7b056648f1ff7cc5376320059d97222a4b1e74a5a81c4829f3d71355ecd" exitCode=0 Jul 16 21:11:54.961411 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.961027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" event={"ID":"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563","Type":"ContainerDied","Data":"c399c7b056648f1ff7cc5376320059d97222a4b1e74a5a81c4829f3d71355ecd"} Jul 16 21:11:54.961411 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:54.961073 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" event={"ID":"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563","Type":"ContainerStarted","Data":"09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01"} Jul 16 21:11:55.377138 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:55.377100 2582 patch_prober.go:28] interesting pod/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 21:11:55.377356 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:55.377155 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 21:11:55.834387 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:55.834338 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:56.151109 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.151077 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:56.252981 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.252950 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca\") pod \"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563\" (UID: \"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563\") " Jul 16 21:11:56.253303 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.253279 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563" (UID: "e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:11:56.354185 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.354150 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:11:56.377394 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.377316 2582 patch_prober.go:28] interesting pod/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 21:11:56.377394 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.377373 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 21:11:56.977971 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.977940 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" Jul 16 21:11:56.978157 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.977972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr" event={"ID":"e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563","Type":"ContainerDied","Data":"09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01"} Jul 16 21:11:56.978157 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:56.978060 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09869eae600bae4097bac86a838e0f14950868054d43acc28724239b7785af01" Jul 16 21:11:57.377511 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:57.377469 2582 patch_prober.go:28] interesting pod/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 16 21:11:57.377927 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:57.377531 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 16 21:11:58.381842 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:58.381801 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:11:58.382742 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:11:58.382724 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:12:11.036015 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:11.035971 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:12:19.966869 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:19.966833 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:12:19.967355 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:19.967098 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" containerID="cri-o://e51057e0956633227d893c648b8ff0c74dca26b200c4694caedb20613a43a9c3" gracePeriod=30 Jul 16 21:12:20.517116 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:20.517070 2582 patch_prober.go:28] interesting pod/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 21:12:20.517292 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:20.517145 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:12:25.154519 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.154476 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerID="e51057e0956633227d893c648b8ff0c74dca26b200c4694caedb20613a43a9c3" exitCode=0 Jul 16 21:12:25.155263 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.154663 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerDied","Data":"e51057e0956633227d893c648b8ff0c74dca26b200c4694caedb20613a43a9c3"} Jul 16 21:12:25.155263 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.154719 2582 scope.go:117] "RemoveContainer" containerID="50e46d48298f5be985011686b8853e89aa622d502e03f123784ce5d7946a7132" Jul 16 21:12:25.225445 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.225421 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:12:25.327355 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327332 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327486 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327364 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhldc\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327486 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327402 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327631 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327612 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327702 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327616 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:25.327759 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327696 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327759 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327745 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327865 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327779 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327865 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327837 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.327964 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.327866 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs\") pod \"9d60bdc4-a123-48c5-9291-71631a497b1d\" (UID: \"9d60bdc4-a123-48c5-9291-71631a497b1d\") " Jul 16 21:12:25.328076 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328043 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:25.328203 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328179 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.328357 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328238 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.328357 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328186 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:25.328357 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328274 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:25.328624 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.328548 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:12:25.329646 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.329626 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:25.329735 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.329660 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc" (OuterVolumeSpecName: "kube-api-access-bhldc") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "kube-api-access-bhldc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:12:25.329893 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.329874 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 21:12:25.329945 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.329889 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d60bdc4-a123-48c5-9291-71631a497b1d" (UID: "9d60bdc4-a123-48c5-9291-71631a497b1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:12:25.429125 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429101 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429125 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429123 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429266 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429132 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d60bdc4-a123-48c5-9291-71631a497b1d-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429266 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429141 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429266 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429149 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429266 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429157 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhldc\" (UniqueName: \"kubernetes.io/projected/9d60bdc4-a123-48c5-9291-71631a497b1d-kube-api-access-bhldc\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:25.429266 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:25.429165 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d60bdc4-a123-48c5-9291-71631a497b1d-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:26.161623 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.161597 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" Jul 16 21:12:26.162031 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.161625 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh" event={"ID":"9d60bdc4-a123-48c5-9291-71631a497b1d","Type":"ContainerDied","Data":"e6cd131f967dbcb4e6162c2a8b107013805b4b27ef84fedcec31dd281f53d121"} Jul 16 21:12:26.162031 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.161705 2582 scope.go:117] "RemoveContainer" containerID="e51057e0956633227d893c648b8ff0c74dca26b200c4694caedb20613a43a9c3" Jul 16 21:12:26.187300 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.187276 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:12:26.190101 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.190059 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-99f81a-openshift-default-6fb4bc5cd7-rccrh"] Jul 16 21:12:26.722584 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:26.722535 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" path="/var/lib/kubelet/pods/9d60bdc4-a123-48c5-9291-71631a497b1d/volumes" Jul 16 21:12:40.629794 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:40.629758 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:12:46.867520 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:46.867481 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:12:46.867989 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:46.867729 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" containerID="cri-o://b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47" gracePeriod=30 Jul 16 21:12:47.801773 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.801740 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:12:47.802134 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802119 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563" containerName="revoke-keys" Jul 16 21:12:47.802179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802136 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563" containerName="revoke-keys" Jul 16 21:12:47.802179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802145 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.802179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802151 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.802179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802167 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.802179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802173 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.802333 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802237 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563" containerName="revoke-keys" Jul 16 21:12:47.802333 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802246 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.802333 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.802253 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d60bdc4-a123-48c5-9291-71631a497b1d" containerName="istio-proxy" Jul 16 21:12:47.806759 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.806741 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.810740 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.810716 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-09402f-openshift-default-dockercfg-m6xl8\"" Jul 16 21:12:47.823459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.823429 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:12:47.927919 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.927888 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.927935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.927959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.927979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfz7v\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.928004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.928146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.928175 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.928194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:47.928329 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:47.928210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029345 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029345 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfz7v\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029527 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029587 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029620 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029611 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029944 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029653 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029944 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.029944 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.029835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.030105 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.030063 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.030270 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.030240 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.030407 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.030364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.031966 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.031936 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.032152 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.032135 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.038134 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.038100 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.038297 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.038276 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfz7v\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v\") pod \"e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.117698 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.117542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:48.285881 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.285855 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:12:48.287863 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:12:48.287837 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d221f87_b4d2_4f18_86fc_63aa48f43aff.slice/crio-0ee343c4aee34b451806b88f6538bbd9625eb2695b1a1387ac44e329474e6fe0 WatchSource:0}: Error finding container 0ee343c4aee34b451806b88f6538bbd9625eb2695b1a1387ac44e329474e6fe0: Status 404 returned error can't find the container with id 0ee343c4aee34b451806b88f6538bbd9625eb2695b1a1387ac44e329474e6fe0 Jul 16 21:12:48.289899 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.289860 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:48.289994 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.289929 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:48.289994 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:48.289955 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:49.299475 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:49.299439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerStarted","Data":"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576"} Jul 16 21:12:49.299899 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:49.299490 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerStarted","Data":"0ee343c4aee34b451806b88f6538bbd9625eb2695b1a1387ac44e329474e6fe0"} Jul 16 21:12:49.319028 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:49.318978 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podStartSLOduration=2.3189627919999998 podStartE2EDuration="2.318962792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:12:49.317273756 +0000 UTC m=+2377.159124885" watchObservedRunningTime="2026-07-16 21:12:49.318962792 +0000 UTC m=+2377.160813988" Jul 16 21:12:50.118285 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:50.118242 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:50.123321 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:50.123297 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:50.306048 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:50.306015 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:50.306994 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:50.306968 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:12:52.133397 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.133326 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:12:52.166125 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166090 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166141 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166186 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166275 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166248 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166434 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166274 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166434 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166300 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166434 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166338 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95x47\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166434 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166407 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166618 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166456 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket\") pod \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\" (UID: \"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae\") " Jul 16 21:12:52.166618 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166500 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:12:52.166757 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166729 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:52.166822 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166789 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.166895 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.166851 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:52.167029 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.167007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:52.167710 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.167683 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:52.168435 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.168414 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:12:52.168557 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.168540 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 21:12:52.168691 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.168670 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47" (OuterVolumeSpecName: "kube-api-access-95x47") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "kube-api-access-95x47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:12:52.169106 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.169087 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" (UID: "2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:12:52.267617 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267555 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267617 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267612 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267617 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267623 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267632 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267640 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267647 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267655 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.267868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.267664 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95x47\" (UniqueName: \"kubernetes.io/projected/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae-kube-api-access-95x47\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:12:52.318327 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.318292 2582 generic.go:356] "Generic (PLEG): container finished" podID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerID="b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47" exitCode=0 Jul 16 21:12:52.318517 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.318335 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" Jul 16 21:12:52.318517 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.318384 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerDied","Data":"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47"} Jul 16 21:12:52.318517 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.318436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg" event={"ID":"2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae","Type":"ContainerDied","Data":"9099d8203eee7683a63c6c6d2fe35f52ee1e0cd402c37d4c05f49cbc5d898339"} Jul 16 21:12:52.318517 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.318466 2582 scope.go:117] "RemoveContainer" containerID="b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47" Jul 16 21:12:52.328720 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.328700 2582 scope.go:117] "RemoveContainer" containerID="2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c" Jul 16 21:12:52.337778 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.337751 2582 scope.go:117] "RemoveContainer" containerID="b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47" Jul 16 21:12:52.338381 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:12:52.338241 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47\": container with ID starting with b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47 not found: ID does not exist" containerID="b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47" Jul 16 21:12:52.338381 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.338283 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47"} err="failed to get container status \"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47\": rpc error: code = NotFound desc = could not find container \"b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47\": container with ID starting with b6feac1f6d35ab594c69bf5aca91ebf990093a468ddb1013c73c121508463b47 not found: ID does not exist" Jul 16 21:12:52.338381 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.338316 2582 scope.go:117] "RemoveContainer" containerID="2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c" Jul 16 21:12:52.338706 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:12:52.338677 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c\": container with ID starting with 2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c not found: ID does not exist" containerID="2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c" Jul 16 21:12:52.338759 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.338717 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c"} err="failed to get container status \"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c\": rpc error: code = NotFound desc = could not find container \"2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c\": container with ID starting with 2571a14e9c1704763b259a797b6d3c9a4c05b5ce682161807ce05f3ed5ffec6c not found: ID does not exist" Jul 16 21:12:52.341446 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.341425 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:12:52.344089 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.344066 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a896c7-openshift-default-6b9768d6b9-68zpg"] Jul 16 21:12:52.723072 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:52.723032 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" path="/var/lib/kubelet/pods/2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae/volumes" Jul 16 21:12:55.933932 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:55.933896 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:12:56.465993 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.465956 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:12:56.466371 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.466357 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:12:56.466418 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.466373 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:12:56.466461 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.466451 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:12:56.471200 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.471177 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.473736 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.473704 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-69d8e7-openshift-default-dockercfg-8s824\"" Jul 16 21:12:56.483424 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.483388 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:12:56.607834 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.607790 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.607834 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.607832 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608088 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.607910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwlhk\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608088 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.607946 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608088 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.607978 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608258 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.608095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608258 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.608134 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608258 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.608204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.608258 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.608232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709644 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709644 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwlhk\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.709973 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.709926 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.710411 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.710108 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.710411 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.710123 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.710411 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.710205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.710539 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.710447 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.710708 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.710682 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.712186 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.712164 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.712388 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.712371 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.717584 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.717498 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.718042 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.718020 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwlhk\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk\") pod \"e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.783653 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.783610 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:56.968852 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.968787 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:12:56.969966 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:12:56.969936 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575b40aa_7fe7_418d_8b0d_12c46fa57e09.slice/crio-7541d5499c94b07a7f48aa857dc2f7ec1592380ecbe6e21e7111926160633f88 WatchSource:0}: Error finding container 7541d5499c94b07a7f48aa857dc2f7ec1592380ecbe6e21e7111926160633f88: Status 404 returned error can't find the container with id 7541d5499c94b07a7f48aa857dc2f7ec1592380ecbe6e21e7111926160633f88 Jul 16 21:12:56.972206 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.972167 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:56.972304 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.972252 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:56.972304 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:56.972297 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:12:57.352882 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:57.352838 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerStarted","Data":"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb"} Jul 16 21:12:57.353095 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:57.352901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerStarted","Data":"7541d5499c94b07a7f48aa857dc2f7ec1592380ecbe6e21e7111926160633f88"} Jul 16 21:12:57.376952 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:57.376884 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podStartSLOduration=1.376860422 podStartE2EDuration="1.376860422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:12:57.371957552 +0000 UTC m=+2385.213808692" watchObservedRunningTime="2026-07-16 21:12:57.376860422 +0000 UTC m=+2385.218711550" Jul 16 21:12:57.784246 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:57.784217 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:58.790047 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:58.790012 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:59.365347 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:59.365292 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:12:59.366385 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:12:59.366366 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:13:03.392623 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.392586 2582 generic.go:356] "Generic (PLEG): container finished" podID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerID="dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576" exitCode=0 Jul 16 21:13:03.393070 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.392653 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerDied","Data":"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576"} Jul 16 21:13:03.393070 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.393066 2582 scope.go:117] "RemoveContainer" containerID="dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576" Jul 16 21:13:03.394095 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.394049 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:03.394153 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.394134 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:03.394194 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:03.394167 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:04.403900 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:04.403870 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerStarted","Data":"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781"} Jul 16 21:13:05.055320 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:05.055280 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:13:05.118534 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:05.118496 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:13:05.134598 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:05.119663 2582 patch_prober.go:28] interesting pod/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 21:13:05.134913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:05.134870 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 21:13:05.306182 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:05.306143 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:13:06.118511 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:06.118464 2582 patch_prober.go:28] interesting pod/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 21:13:06.119020 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:06.118526 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 21:13:07.118307 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:07.118268 2582 patch_prober.go:28] interesting pod/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 21:13:07.118487 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:07.118332 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 21:13:08.119020 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:08.118985 2582 patch_prober.go:28] interesting pod/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 16 21:13:08.119483 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:08.119453 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 16 21:13:09.128354 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:09.128311 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:13:09.131836 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:09.131800 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:13:12.464368 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.464333 2582 generic.go:356] "Generic (PLEG): container finished" podID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerID="e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb" exitCode=0 Jul 16 21:13:12.464905 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.464416 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerDied","Data":"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb"} Jul 16 21:13:12.464979 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.464928 2582 scope.go:117] "RemoveContainer" containerID="e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb" Jul 16 21:13:12.466008 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.465979 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:12.466116 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.466094 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:12.466169 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.466152 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 16 21:13:12.860932 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.860890 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:13:12.865670 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.865637 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:13:12.877911 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.877891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:13:12.882449 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:12.882431 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:13:13.474312 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:13.474275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerStarted","Data":"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296"} Jul 16 21:13:13.784387 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:13.784362 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:13:13.785743 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:13.785715 2582 patch_prober.go:28] interesting pod/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 21:13:13.785911 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:13.785770 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 21:13:14.366258 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:14.366222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:13:14.784903 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:14.784873 2582 patch_prober.go:28] interesting pod/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 21:13:14.785337 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:14.784931 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 21:13:15.784937 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:15.784895 2582 patch_prober.go:28] interesting pod/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 21:13:15.785373 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:15.784958 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 21:13:16.784868 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:16.784830 2582 patch_prober.go:28] interesting pod/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 16 21:13:16.785063 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:16.784900 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 16 21:13:17.788044 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:17.788013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:13:17.789016 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:17.788992 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:13:18.233415 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:18.233332 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:13:34.735943 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:34.735863 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:13:50.744617 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:13:50.744563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:14:04.731659 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:14:04.731625 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:14:22.028851 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:14:22.028811 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:14:40.236713 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:14:40.236678 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:14:57.131062 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:14:57.131028 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:15:00.156626 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.156587 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks"] Jul 16 21:15:00.157040 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.157011 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:15:00.157040 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.157024 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:15:00.157129 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.157104 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b89e77f-33ea-4ea9-9a7e-a87a16bd99ae" containerName="istio-proxy" Jul 16 21:15:00.160331 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.160314 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" Jul 16 21:15:00.162985 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.162953 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fcvh8\"" Jul 16 21:15:00.171013 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.170990 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" Jul 16 21:15:00.178534 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.178505 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks"] Jul 16 21:15:00.312045 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:00.312012 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks"] Jul 16 21:15:00.314846 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:15:00.314820 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef099294_572f_44f3_885d_59819c89b9d0.slice/crio-889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e WatchSource:0}: Error finding container 889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e: Status 404 returned error can't find the container with id 889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e Jul 16 21:15:01.131181 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:01.131145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" event={"ID":"ef099294-572f-44f3-885d-59819c89b9d0","Type":"ContainerStarted","Data":"775e765df37bf54adf0f7fdca55b1830cea3a695ef2f78802310625b13637cbd"} Jul 16 21:15:01.131363 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:01.131197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" event={"ID":"ef099294-572f-44f3-885d-59819c89b9d0","Type":"ContainerStarted","Data":"889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e"} Jul 16 21:15:01.163643 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:01.163592 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" podStartSLOduration=1.163563904 podStartE2EDuration="1.163563904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:01.159839246 +0000 UTC m=+2509.001690374" watchObservedRunningTime="2026-07-16 21:15:01.163563904 +0000 UTC m=+2509.005415031" Jul 16 21:15:02.138897 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:02.138862 2582 generic.go:356] "Generic (PLEG): container finished" podID="ef099294-572f-44f3-885d-59819c89b9d0" containerID="775e765df37bf54adf0f7fdca55b1830cea3a695ef2f78802310625b13637cbd" exitCode=0 Jul 16 21:15:02.139078 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:02.138935 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" event={"ID":"ef099294-572f-44f3-885d-59819c89b9d0","Type":"ContainerDied","Data":"775e765df37bf54adf0f7fdca55b1830cea3a695ef2f78802310625b13637cbd"} Jul 16 21:15:03.280443 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:03.280416 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" Jul 16 21:15:04.150544 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:04.150512 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" Jul 16 21:15:04.150544 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:04.150544 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks" event={"ID":"ef099294-572f-44f3-885d-59819c89b9d0","Type":"ContainerDied","Data":"889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e"} Jul 16 21:15:04.150782 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:04.150612 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="889503bedb6c239a11fe5ad164376c4a22898219629f1259bd31ed127d61fe0e" Jul 16 21:15:15.536047 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:15.536005 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:15:29.836410 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:29.836374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:15:42.630864 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:15:42.630828 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:16:06.238195 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:16:06.238156 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:16:37.334748 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:16:37.334659 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:16:51.244839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:16:51.244801 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:17:09.536521 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:17:09.536471 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:17:27.544592 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:17:27.544541 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:17:55.541476 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:17:55.541435 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:18:12.907397 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:12.907367 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:18:12.911826 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:12.911802 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:18:12.919364 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:12.919342 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:18:12.923045 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:12.923026 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r5n26_8889d89f-ec52-4033-b2ca-58192f929616/ovn-acl-logging/0.log" Jul 16 21:18:15.345057 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:15.345021 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:18:32.846234 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:32.846196 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:18:49.050596 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:18:49.050523 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:19:11.549313 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:11.549277 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks"] Jul 16 21:19:11.555864 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:11.555836 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-09402f-29737275-s6cks"] Jul 16 21:19:12.723528 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:12.723495 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef099294-572f-44f3-885d-59819c89b9d0" path="/var/lib/kubelet/pods/ef099294-572f-44f3-885d-59819c89b9d0/volumes" Jul 16 21:19:14.646579 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:14.646535 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:19:23.943016 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:23.942984 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:19:23.943438 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:23.943188 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" containerID="cri-o://7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781" gracePeriod=30 Jul 16 21:19:24.131963 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:24.131921 2582 patch_prober.go:28] interesting pod/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 21:19:24.132128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:24.131984 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:19:29.200385 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.200357 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:19:29.310822 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.310793 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.310988 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.310850 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.310988 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.310882 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311113 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311095 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311170 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311110 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:29.311170 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311129 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311170 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311138 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:19:29.311170 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311147 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311385 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311221 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfz7v\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311385 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311328 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:29.311479 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311277 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311479 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311427 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket\") pod \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\" (UID: \"4d221f87-b4d2-4f18-86fc-63aa48f43aff\") " Jul 16 21:19:29.311715 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311678 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:29.311809 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311793 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.311871 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311814 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.311871 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311829 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.311871 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.311843 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.312118 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.312096 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:29.313167 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.313143 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 21:19:29.313326 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.313302 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v" (OuterVolumeSpecName: "kube-api-access-tfz7v") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "kube-api-access-tfz7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:19:29.313566 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.313542 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:19:29.313566 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.313545 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d221f87-b4d2-4f18-86fc-63aa48f43aff" (UID: "4d221f87-b4d2-4f18-86fc-63aa48f43aff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:29.412628 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.412607 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.412628 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.412626 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.412766 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.412635 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.412766 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.412644 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d221f87-b4d2-4f18-86fc-63aa48f43aff-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.412766 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.412653 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfz7v\" (UniqueName: \"kubernetes.io/projected/4d221f87-b4d2-4f18-86fc-63aa48f43aff-kube-api-access-tfz7v\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:29.723397 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.723317 2582 generic.go:356] "Generic (PLEG): container finished" podID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerID="7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781" exitCode=0 Jul 16 21:19:29.723397 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.723361 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" Jul 16 21:19:29.723397 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.723377 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerDied","Data":"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781"} Jul 16 21:19:29.723694 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.723428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn" event={"ID":"4d221f87-b4d2-4f18-86fc-63aa48f43aff","Type":"ContainerDied","Data":"0ee343c4aee34b451806b88f6538bbd9625eb2695b1a1387ac44e329474e6fe0"} Jul 16 21:19:29.723694 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.723460 2582 scope.go:117] "RemoveContainer" containerID="7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781" Jul 16 21:19:29.734068 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.734052 2582 scope.go:117] "RemoveContainer" containerID="dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576" Jul 16 21:19:29.742338 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.742322 2582 scope.go:117] "RemoveContainer" containerID="7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781" Jul 16 21:19:29.742628 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:19:29.742608 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781\": container with ID starting with 7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781 not found: ID does not exist" containerID="7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781" Jul 16 21:19:29.742671 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.742636 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781"} err="failed to get container status \"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781\": rpc error: code = NotFound desc = could not find container \"7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781\": container with ID starting with 7e81da4636fbb1032f983d409144c9df845e8f857e1118ed1eec32c5b75f9781 not found: ID does not exist" Jul 16 21:19:29.742671 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.742654 2582 scope.go:117] "RemoveContainer" containerID="dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576" Jul 16 21:19:29.742866 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:19:29.742850 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576\": container with ID starting with dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576 not found: ID does not exist" containerID="dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576" Jul 16 21:19:29.742907 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.742872 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576"} err="failed to get container status \"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576\": rpc error: code = NotFound desc = could not find container \"dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576\": container with ID starting with dba6f37e635ddc9a7431c4d6d3d33a0b992ef376f3e4481c177c012136d2f576 not found: ID does not exist" Jul 16 21:19:29.750448 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.750426 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:19:29.758647 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:29.758627 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-09402f-openshift-default-7c6ff9487-zggmn"] Jul 16 21:19:30.722110 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:30.722077 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" path="/var/lib/kubelet/pods/4d221f87-b4d2-4f18-86fc-63aa48f43aff/volumes" Jul 16 21:19:37.430939 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:37.430842 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4b72t"] Jul 16 21:19:46.214618 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:46.214584 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:19:46.215127 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:46.214828 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" containerID="cri-o://84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296" gracePeriod=30 Jul 16 21:19:47.790942 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:47.790896 2582 patch_prober.go:28] interesting pod/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 21:19:47.791320 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:47.790977 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:19:51.477281 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.477198 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:19:51.512014 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.511981 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.512210 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512049 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.512210 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512079 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwlhk\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.512210 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512104 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.512383 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512357 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512592 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512678 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512705 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512733 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512747 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.512763 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo\") pod \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\" (UID: \"575b40aa-7fe7-418d-8b0d-12c46fa57e09\") " Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.513014 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.513029 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.513721 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.513746 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data" (OuterVolumeSpecName: "istio-data") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.513780 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:51.514913 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.514864 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:19:51.515645 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.515566 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk" (OuterVolumeSpecName: "kube-api-access-wwlhk") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "kube-api-access-wwlhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:19:51.515845 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.515825 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 21:19:51.516360 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.516342 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token" (OuterVolumeSpecName: "istio-token") pod "575b40aa-7fe7-418d-8b0d-12c46fa57e09" (UID: "575b40aa-7fe7-418d-8b0d-12c46fa57e09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:19:51.613961 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613927 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.613961 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613956 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.613961 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613966 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.614202 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613974 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.614202 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613986 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.614202 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.613995 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwlhk\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-kube-api-access-wwlhk\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.614202 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.614005 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575b40aa-7fe7-418d-8b0d-12c46fa57e09-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 16 21:19:51.850780 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.850744 2582 generic.go:356] "Generic (PLEG): container finished" podID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerID="84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296" exitCode=0 Jul 16 21:19:51.850949 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.850810 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" Jul 16 21:19:51.850949 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.850827 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerDied","Data":"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296"} Jul 16 21:19:51.850949 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.850906 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp" event={"ID":"575b40aa-7fe7-418d-8b0d-12c46fa57e09","Type":"ContainerDied","Data":"7541d5499c94b07a7f48aa857dc2f7ec1592380ecbe6e21e7111926160633f88"} Jul 16 21:19:51.850949 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.850929 2582 scope.go:117] "RemoveContainer" containerID="84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296" Jul 16 21:19:51.860927 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.860909 2582 scope.go:117] "RemoveContainer" containerID="e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb" Jul 16 21:19:51.870047 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.870027 2582 scope.go:117] "RemoveContainer" containerID="84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296" Jul 16 21:19:51.870325 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:19:51.870302 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296\": container with ID starting with 84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296 not found: ID does not exist" containerID="84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296" Jul 16 21:19:51.870366 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.870335 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296"} err="failed to get container status \"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296\": rpc error: code = NotFound desc = could not find container \"84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296\": container with ID starting with 84edb033747dd1091f391f6a35c03f5235b8e64e9a3a1b31ee1c080aeb9d5296 not found: ID does not exist" Jul 16 21:19:51.870366 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.870353 2582 scope.go:117] "RemoveContainer" containerID="e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb" Jul 16 21:19:51.870645 ip-10-0-135-240 kubenswrapper[2582]: E0716 21:19:51.870627 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb\": container with ID starting with e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb not found: ID does not exist" containerID="e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb" Jul 16 21:19:51.870692 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.870649 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb"} err="failed to get container status \"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb\": rpc error: code = NotFound desc = could not find container \"e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb\": container with ID starting with e93b538283027c901a3c8226434f9ad106d1c3bf4f3e06fe50370c27c4caefeb not found: ID does not exist" Jul 16 21:19:51.874473 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.874448 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:19:51.878695 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:51.878669 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-69d8e7-openshift-default-7d8c69b775-gb8mp"] Jul 16 21:19:52.723184 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:52.723151 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" path="/var/lib/kubelet/pods/575b40aa-7fe7-418d-8b0d-12c46fa57e09/volumes" Jul 16 21:19:53.328466 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:53.328431 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6477549b-rjlrs_f0335755-dcf1-4f86-b7ba-f39eb260d493/copy-manifests/0.log" Jul 16 21:19:53.336245 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:53.336220 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6477549b-rjlrs_f0335755-dcf1-4f86-b7ba-f39eb260d493/manager/0.log" Jul 16 21:19:53.821950 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:53.821917 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737245-pwmkp_fd4c89b1-49b3-46fd-95dc-4883d67127f1/cleanup/0.log" Jul 16 21:19:53.937630 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:53.937597 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737260-sfbmm_273f2cf5-413c-43a2-9108-6d4548e58334/cleanup/0.log" Jul 16 21:19:54.163090 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.163018 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-25df27a3-wz5q7_d9d8c42e-8b04-4861-9160-768538ba7a20/revoke-keys/0.log" Jul 16 21:19:54.283911 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.283888 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-866d7766-svhhx_4baf2db1-91dc-4c5b-b211-b1e1b0b94a35/revoke-keys/0.log" Jul 16 21:19:54.402948 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.402919 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-99f81a-vwlgr_e2ff80f3-7854-4f7a-93c2-f7a0dc8a0563/revoke-keys/0.log" Jul 16 21:19:54.625271 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.625238 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-f05f95fc-242l6_fa01a427-576c-4d71-abc2-2e5950cb999c/revoke-keys/0.log" Jul 16 21:19:54.737371 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.737345 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-60ce31-qx2q4_cd294a37-7a16-4cdb-be61-633ef44ad42b/revoke-keys/0.log" Jul 16 21:19:54.850551 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.850523 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-054dd3-22nfs_4c927d5e-3674-4449-8d7f-323a52ca753d/revoke-keys/0.log" Jul 16 21:19:54.958931 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:54.958856 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-18adf534-nccd5_09520c6b-bde6-4660-bac9-937fd8e8fd23/revoke-keys/0.log" Jul 16 21:19:55.079905 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:55.079876 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-27a1e9ed-nkh2b_6a0f41dc-a3ef-4cd3-94ed-9616b5b4db25/revoke-keys/0.log" Jul 16 21:19:55.193326 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:55.193297 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-a3e438d9-whnlh_0583efc8-8ab2-4a0e-8cdf-5bf731e87b57/revoke-keys/0.log" Jul 16 21:19:56.332038 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.332009 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/extract/0.log" Jul 16 21:19:56.338739 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.338719 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/util/0.log" Jul 16 21:19:56.345776 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.345758 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/pull/0.log" Jul 16 21:19:56.460952 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.460926 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/util/0.log" Jul 16 21:19:56.470270 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.470247 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/pull/0.log" Jul 16 21:19:56.477134 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.477115 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/extract/0.log" Jul 16 21:19:56.595630 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.595507 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/util/0.log" Jul 16 21:19:56.602790 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.602761 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/pull/0.log" Jul 16 21:19:56.610022 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.610001 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/extract/0.log" Jul 16 21:19:56.730179 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.730149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/util/0.log" Jul 16 21:19:56.737719 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.737692 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/pull/0.log" Jul 16 21:19:56.744469 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:56.744446 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/extract/0.log" Jul 16 21:19:57.579137 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:57.579114 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4b72t_224fae5f-8d3c-4711-aaa2-f2317f0be9fb/limitador/0.log" Jul 16 21:19:58.202435 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:58.202407 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7vrjf_d14b7f24-e9ce-4b65-a89e-53843aadef8d/discovery/0.log" Jul 16 21:19:58.908361 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:58.908325 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-665df85b66-nl88f_c70ee74a-e1b1-4c18-a631-cb4e6dd97804/router/0.log" Jul 16 21:19:59.708739 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:19:59.708712 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hbwp5_21abb431-60d7-4139-ada2-05bfbd36d4a0/main/0.log" Jul 16 21:20:05.203412 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203373 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rmclr/must-gather-qvhmh"] Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203722 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef099294-572f-44f3-885d-59819c89b9d0" containerName="cleanup" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203733 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef099294-572f-44f3-885d-59819c89b9d0" containerName="cleanup" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203747 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203752 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203763 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203769 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203775 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203780 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203794 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.203839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203801 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.204128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203856 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.204128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203863 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.204128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203874 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef099294-572f-44f3-885d-59819c89b9d0" containerName="cleanup" Jul 16 21:20:05.204128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203969 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="575b40aa-7fe7-418d-8b0d-12c46fa57e09" containerName="istio-proxy" Jul 16 21:20:05.204128 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.203974 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d221f87-b4d2-4f18-86fc-63aa48f43aff" containerName="istio-proxy" Jul 16 21:20:05.208536 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.208516 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.211158 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.211134 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rmclr\"/\"kube-root-ca.crt\"" Jul 16 21:20:05.211274 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.211255 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rmclr\"/\"default-dockercfg-g7srr\"" Jul 16 21:20:05.212143 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.212125 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rmclr\"/\"openshift-service-ca.crt\"" Jul 16 21:20:05.216636 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.216367 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rmclr/must-gather-qvhmh"] Jul 16 21:20:05.336851 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.336803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-must-gather-output\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.337029 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.336881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzrk5\" (UniqueName: \"kubernetes.io/projected/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-kube-api-access-lzrk5\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.438297 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.438247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-must-gather-output\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.438502 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.438310 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzrk5\" (UniqueName: \"kubernetes.io/projected/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-kube-api-access-lzrk5\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.438696 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.438676 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-must-gather-output\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.447669 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.447640 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzrk5\" (UniqueName: \"kubernetes.io/projected/0bd930e0-baa2-421c-8e5d-b4d51ea8bc59-kube-api-access-lzrk5\") pod \"must-gather-qvhmh\" (UID: \"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59\") " pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.519687 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.519591 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rmclr/must-gather-qvhmh" Jul 16 21:20:05.666531 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.666500 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rmclr/must-gather-qvhmh"] Jul 16 21:20:05.667997 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:20:05.667963 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bd930e0_baa2_421c_8e5d_b4d51ea8bc59.slice/crio-b5fa5d502846c9ff78c4f902bad50ede2639958ddb16b8437c5fa0749258fc5d WatchSource:0}: Error finding container b5fa5d502846c9ff78c4f902bad50ede2639958ddb16b8437c5fa0749258fc5d: Status 404 returned error can't find the container with id b5fa5d502846c9ff78c4f902bad50ede2639958ddb16b8437c5fa0749258fc5d Jul 16 21:20:05.670228 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.670208 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:20:05.932002 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:05.931966 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rmclr/must-gather-qvhmh" event={"ID":"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59","Type":"ContainerStarted","Data":"b5fa5d502846c9ff78c4f902bad50ede2639958ddb16b8437c5fa0749258fc5d"} Jul 16 21:20:06.941584 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:06.941478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rmclr/must-gather-qvhmh" event={"ID":"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59","Type":"ContainerStarted","Data":"dba14cb93700edb2f3165f945a6ba1f6c1c532711cf0badba4e95eaa0a55f970"} Jul 16 21:20:06.941584 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:06.941527 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rmclr/must-gather-qvhmh" event={"ID":"0bd930e0-baa2-421c-8e5d-b4d51ea8bc59","Type":"ContainerStarted","Data":"f666592dcf1e0f37b792d9c78b232744b94122db3e2a7e7aa16a6c8c82aaca1a"} Jul 16 21:20:06.960210 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:06.960136 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rmclr/must-gather-qvhmh" podStartSLOduration=1.072530315 podStartE2EDuration="1.96010827s" totalImagesPullingTime="887.577955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:06.95681954 +0000 UTC m=+2814.798670678" watchObservedRunningTime="2026-07-16 21:20:06.96010827 +0000 UTC m=+2814.801959591" Jul 16 21:20:08.144905 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:08.144811 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-84n8n_5556d69e-b310-4298-888e-5f47489a878b/global-pull-secret-syncer/0.log" Jul 16 21:20:08.416839 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:08.416752 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wgj47_3a4ed8fd-86de-40e9-b633-280c9b429ec2/konnectivity-agent/0.log" Jul 16 21:20:08.481556 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:08.481523 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-240.ec2.internal_9f3a70c0e179a36534f61d6399c4980f/haproxy/0.log" Jul 16 21:20:13.470654 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.470528 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/extract/0.log" Jul 16 21:20:13.499037 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.499007 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/util/0.log" Jul 16 21:20:13.531592 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.531060 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759plx7t_8aedf5aa-f093-4d8c-b259-357f945cc650/pull/0.log" Jul 16 21:20:13.563844 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.563812 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/extract/0.log" Jul 16 21:20:13.594487 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.594460 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/util/0.log" Jul 16 21:20:13.621017 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.620989 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nv479_42e6ea38-7f11-4896-9dd3-7acecc7912da/pull/0.log" Jul 16 21:20:13.649810 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.649777 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/extract/0.log" Jul 16 21:20:13.670774 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.670744 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/util/0.log" Jul 16 21:20:13.701169 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.701137 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g8gbh_f73674d0-4ace-4c85-833a-32e97edec1fe/pull/0.log" Jul 16 21:20:13.739482 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.739405 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/extract/0.log" Jul 16 21:20:13.774433 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.774392 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/util/0.log" Jul 16 21:20:13.798823 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:13.798797 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lx9gn_895d270b-18d6-498b-9bc0-c56b796f9683/pull/0.log" Jul 16 21:20:14.586093 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:14.586066 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4b72t_224fae5f-8d3c-4711-aaa2-f2317f0be9fb/limitador/0.log" Jul 16 21:20:16.540020 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:16.539985 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-9jdqp_41e2fac7-5f11-405f-aa74-f4241008bf24/cluster-monitoring-operator/0.log" Jul 16 21:20:16.647358 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:16.647310 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76bff8ddc6-dr2lk_90864d69-aee5-46e7-9a37-fb46c3d973f4/metrics-server/0.log" Jul 16 21:20:16.705493 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:16.705460 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zpk7_4508933c-f7d9-4188-ace2-b11218728255/node-exporter/0.log" Jul 16 21:20:16.727643 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:16.727612 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zpk7_4508933c-f7d9-4188-ace2-b11218728255/kube-rbac-proxy/0.log" Jul 16 21:20:16.749869 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:16.749837 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zpk7_4508933c-f7d9-4188-ace2-b11218728255/init-textfile/0.log" Jul 16 21:20:17.390514 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.390488 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/thanos-query/0.log" Jul 16 21:20:17.413423 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.413391 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/kube-rbac-proxy-web/0.log" Jul 16 21:20:17.435457 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.435366 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/kube-rbac-proxy/0.log" Jul 16 21:20:17.456340 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.456317 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/prom-label-proxy/0.log" Jul 16 21:20:17.479737 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.479689 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/kube-rbac-proxy-rules/0.log" Jul 16 21:20:17.500503 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:17.500476 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f7c6678f8-2btww_1614543f-e339-48d8-8f06-cd4beb611930/kube-rbac-proxy-metrics/0.log" Jul 16 21:20:18.316231 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.316193 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt"] Jul 16 21:20:18.322490 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.322452 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.336733 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.336698 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt"] Jul 16 21:20:18.483213 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.483181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkmmp\" (UniqueName: \"kubernetes.io/projected/17ca05e3-85aa-47b0-9708-22ce7266501b-kube-api-access-tkmmp\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.483391 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.483238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-podres\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.483391 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.483321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-proc\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.483391 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.483383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-sys\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.483520 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.483408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-lib-modules\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584768 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkmmp\" (UniqueName: \"kubernetes.io/projected/17ca05e3-85aa-47b0-9708-22ce7266501b-kube-api-access-tkmmp\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584768 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-podres\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584984 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-proc\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584984 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584857 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-sys\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584984 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-lib-modules\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.584984 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-podres\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.585162 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.584985 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-proc\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.585162 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.585023 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-sys\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.585162 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.585043 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17ca05e3-85aa-47b0-9708-22ce7266501b-lib-modules\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.592682 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.592661 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkmmp\" (UniqueName: \"kubernetes.io/projected/17ca05e3-85aa-47b0-9708-22ce7266501b-kube-api-access-tkmmp\") pod \"perf-node-gather-daemonset-pb5bt\" (UID: \"17ca05e3-85aa-47b0-9708-22ce7266501b\") " pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.596994 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.596974 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-ztxn5_d839394e-abbe-4b8b-8a60-e52d3fe9f13d/networking-console-plugin/0.log" Jul 16 21:20:18.637453 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.637425 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:18.823714 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:18.823670 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt"] Jul 16 21:20:18.824423 ip-10-0-135-240 kubenswrapper[2582]: W0716 21:20:18.824388 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod17ca05e3_85aa_47b0_9708_22ce7266501b.slice/crio-678cf04973e986a92eb63cb234079cf581db980cc92b839ba336f76c4d140183 WatchSource:0}: Error finding container 678cf04973e986a92eb63cb234079cf581db980cc92b839ba336f76c4d140183: Status 404 returned error can't find the container with id 678cf04973e986a92eb63cb234079cf581db980cc92b839ba336f76c4d140183 Jul 16 21:20:19.052804 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.052763 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" event={"ID":"17ca05e3-85aa-47b0-9708-22ce7266501b","Type":"ContainerStarted","Data":"9535df19c66803702328645d6ed78dd247c69d1ed80a79c604f879d11e648e71"} Jul 16 21:20:19.053000 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.052828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" event={"ID":"17ca05e3-85aa-47b0-9708-22ce7266501b","Type":"ContainerStarted","Data":"678cf04973e986a92eb63cb234079cf581db980cc92b839ba336f76c4d140183"} Jul 16 21:20:19.053098 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.053071 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:19.073524 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.073474 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" podStartSLOduration=1.07346008 podStartE2EDuration="1.07346008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:19.069936081 +0000 UTC m=+2826.911787208" watchObservedRunningTime="2026-07-16 21:20:19.07346008 +0000 UTC m=+2826.915311206" Jul 16 21:20:19.173754 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.173675 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/1.log" Jul 16 21:20:19.181906 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.181873 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fdc7s_35dd9ae8-1456-44ff-8b64-5790858c897c/console-operator/2.log" Jul 16 21:20:19.633440 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:19.633415 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d4c586979-m7r8b_96307a7f-010b-4845-8fab-c7c196623fbb/console/0.log" Jul 16 21:20:20.217256 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:20.217213 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qcqxh_3d410c6a-51fe-4d68-bc58-d72bbca0e752/volume-data-source-validator/0.log" Jul 16 21:20:21.321843 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:21.321816 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x958g_4fc407f8-7328-46c2-bbf9-53e6bae72070/dns/0.log" Jul 16 21:20:21.348773 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:21.348746 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x958g_4fc407f8-7328-46c2-bbf9-53e6bae72070/kube-rbac-proxy/0.log" Jul 16 21:20:21.397477 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:21.397450 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gvsbw_b9b15667-eba3-4608-a2f7-2c955b9705dc/dns-node-resolver/0.log" Jul 16 21:20:21.988277 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:21.988252 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-npv56_5d55b6a6-cbd1-41e1-97fc-80c64efeca7b/node-ca/0.log" Jul 16 21:20:23.132928 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:23.132896 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7vrjf_d14b7f24-e9ce-4b65-a89e-53843aadef8d/discovery/0.log" Jul 16 21:20:23.332151 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:23.332122 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-665df85b66-nl88f_c70ee74a-e1b1-4c18-a631-cb4e6dd97804/router/0.log" Jul 16 21:20:24.031921 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.031891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2z2lm_e6eb2d8f-ddfe-403b-91ea-75cecba86f98/serve-healthcheck-canary/0.log" Jul 16 21:20:24.541544 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.541514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-k7vcw_da4a5426-e1fa-4a86-886b-8415a2614e7e/insights-operator/0.log" Jul 16 21:20:24.543974 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.543953 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-k7vcw_da4a5426-e1fa-4a86-886b-8415a2614e7e/insights-operator/1.log" Jul 16 21:20:24.713459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.713435 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxwv5_bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1/kube-rbac-proxy/0.log" Jul 16 21:20:24.736683 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.736648 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxwv5_bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1/exporter/0.log" Jul 16 21:20:24.757929 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:24.757903 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxwv5_bc55b10a-cf1f-47f8-9c1a-0b2c0d060ea1/extractor/0.log" Jul 16 21:20:25.070619 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:25.070554 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rmclr/perf-node-gather-daemonset-pb5bt" Jul 16 21:20:27.590795 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:27.590765 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6477549b-rjlrs_f0335755-dcf1-4f86-b7ba-f39eb260d493/manager/0.log" Jul 16 21:20:27.613063 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:27.612972 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6477549b-rjlrs_f0335755-dcf1-4f86-b7ba-f39eb260d493/copy-manifests/0.log" Jul 16 21:20:28.905472 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:28.905437 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7vrjf_d14b7f24-e9ce-4b65-a89e-53843aadef8d/discovery/0.log" Jul 16 21:20:29.088949 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:29.088912 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-649bd55f77-wpxkt_08398a93-dbe9-4ed0-8400-11b601c55790/manager/0.log" Jul 16 21:20:29.111072 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:29.111029 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-665df85b66-nl88f_c70ee74a-e1b1-4c18-a631-cb4e6dd97804/router/0.log" Jul 16 21:20:30.558911 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:30.558861 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7vrjf_d14b7f24-e9ce-4b65-a89e-53843aadef8d/discovery/0.log" Jul 16 21:20:30.710459 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:30.710385 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-665df85b66-nl88f_c70ee74a-e1b1-4c18-a631-cb4e6dd97804/router/0.log" Jul 16 21:20:34.705965 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:34.705934 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qsgfs_640a485f-3383-44c9-b7ff-1974fe02179e/kube-storage-version-migrator-operator/1.log" Jul 16 21:20:34.706778 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:34.706752 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qsgfs_640a485f-3383-44c9-b7ff-1974fe02179e/kube-storage-version-migrator-operator/0.log" Jul 16 21:20:35.739279 ip-10-0-135-240 kubenswrapper[2582]: I0716 21:20:35.739203 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5nssp_fe300f32-518a-4ff9-9700-d7ffd99864bd/kube-multus/0.log"