Jul 08 01:40:39.758672 ip-10-0-136-194 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 01:40:39.758681 ip-10-0-136-194 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 01:40:39.758688 ip-10-0-136-194 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 01:40:39.758935 ip-10-0-136-194 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 01:40:49.924604 ip-10-0-136-194 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 01:40:49.924626 ip-10-0-136-194 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0884ce9fc2ab4f959f8c5ca0dc21903a -- Jul 08 01:42:46.682075 ip-10-0-136-194 systemd[1]: Starting Kubernetes Kubelet... Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.013063 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017845 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017856 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017860 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017863 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017866 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017869 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017872 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:47.170053 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017875 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:47.089030 ip-10-0-136-194 systemd[1]: Started Kubernetes Kubelet. Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017879 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017882 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017886 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017889 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017892 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017895 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017898 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017901 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017904 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017907 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017910 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017912 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017917 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017921 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017923 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017926 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017929 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017931 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017934 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:47.171199 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017937 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017939 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017943 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017946 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017948 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017951 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017954 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017957 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017961 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017965 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017968 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017970 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017973 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017976 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017978 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017981 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017983 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017985 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017989 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017991 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:47.171930 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017994 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017997 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.017999 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018002 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018004 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018007 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018010 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018013 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018016 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018018 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018021 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018023 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018028 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018031 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018035 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018037 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018040 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018043 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018046 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:47.172558 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018048 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018051 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018053 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018057 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018059 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018062 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018065 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018067 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018071 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018074 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018077 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018080 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018082 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018086 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018088 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018091 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018094 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018096 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018098 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018102 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:47.173495 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018105 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018108 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018110 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018113 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018116 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018119 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018121 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018124 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018126 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018129 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018131 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018134 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.018137 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020221 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020229 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020233 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020236 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020239 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020242 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:47.174267 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020244 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020247 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020249 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020252 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020254 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020257 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020260 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020262 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020265 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020268 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020272 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020274 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020277 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020281 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020285 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020288 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020291 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020294 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020296 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:47.175013 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020298 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020301 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020304 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020306 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020309 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020311 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020314 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020316 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020319 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020322 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020325 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020329 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020332 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020335 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020339 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020343 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020345 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020349 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020351 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:47.176285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020354 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020357 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020360 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020363 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020365 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020368 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020371 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020373 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020376 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020378 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020381 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020384 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020386 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020389 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020391 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020394 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020396 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020399 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020402 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020405 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:47.177348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020407 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020410 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020413 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020416 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020418 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020420 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020423 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020425 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020428 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020431 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020433 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020435 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020438 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020440 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020443 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020445 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020448 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020450 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020453 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020456 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:49.140871 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020458 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020463 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020466 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020469 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020472 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020474 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020477 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020480 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020483 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020486 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020488 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020491 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020494 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020496 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020499 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020882 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020887 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020890 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020893 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020895 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:49.152490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020898 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020901 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020904 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020906 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020909 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020911 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020914 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020916 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020919 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020922 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020924 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020927 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020930 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020933 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020936 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020939 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020941 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020944 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020947 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020950 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:49.153329 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020953 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020956 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020958 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020961 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020963 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020966 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020968 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020971 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020973 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020976 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020979 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020981 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020984 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020987 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020989 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020992 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020994 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.020998 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021001 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:49.154129 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021004 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021007 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021010 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021012 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021015 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021017 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021024 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021027 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021031 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021035 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021041 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021044 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021046 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021049 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021052 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021054 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021057 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021059 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021062 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021064 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:49.154744 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021067 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021070 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021072 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021075 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021077 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021079 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021082 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021085 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021087 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021090 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021092 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021095 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021098 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021101 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021103 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021106 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021108 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021111 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021115 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021117 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:49.155356 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021120 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021122 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021127 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021129 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021132 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021135 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021137 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021140 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021142 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021146 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021150 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021152 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021155 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021158 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021161 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021232 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021239 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021248 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021252 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021256 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021260 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 01:42:49.155951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021264 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021268 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021271 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021274 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021278 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021281 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021284 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021287 2560 flags.go:64] FLAG: --cgroup-root="" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021291 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021295 2560 flags.go:64] FLAG: --client-ca-file="" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021298 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021301 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021304 2560 flags.go:64] FLAG: --cluster-domain="" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021308 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021312 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021315 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021318 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021322 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021325 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021328 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021331 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021334 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021337 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021340 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 01:42:49.156539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021343 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021346 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021350 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021353 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021356 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021359 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021362 2560 flags.go:64] FLAG: --enable-server="true" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021365 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021369 2560 flags.go:64] FLAG: --event-burst="100" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021372 2560 flags.go:64] FLAG: --event-qps="50" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021375 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021378 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021381 2560 flags.go:64] FLAG: --eviction-hard="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021385 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021388 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021391 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021393 2560 flags.go:64] FLAG: --eviction-soft="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021398 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021401 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021404 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021406 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021411 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021414 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021417 2560 flags.go:64] FLAG: --feature-gates="" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021422 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 01:42:49.157232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021425 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021428 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021431 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021434 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021437 2560 flags.go:64] FLAG: --help="false" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021440 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021443 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021446 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021449 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021452 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021456 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021459 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021461 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021464 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021467 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021470 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021473 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021476 2560 flags.go:64] FLAG: --kube-reserved="" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021478 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021481 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021484 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021486 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021489 2560 flags.go:64] FLAG: --lock-file="" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021492 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 01:42:49.157959 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021496 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021499 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021504 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021507 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021511 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021514 2560 flags.go:64] FLAG: --logging-format="text" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021517 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021520 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021523 2560 flags.go:64] FLAG: --manifest-url="" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021526 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021530 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021533 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021538 2560 flags.go:64] FLAG: --max-pods="110" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021541 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021544 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021547 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021550 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021553 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021556 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021559 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021566 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021569 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021572 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 01:42:49.158592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021575 2560 flags.go:64] FLAG: --pod-cidr="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021578 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021584 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021586 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021589 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021592 2560 flags.go:64] FLAG: --port="10250" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021595 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021599 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f61081bf463b213b" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021602 2560 flags.go:64] FLAG: --qos-reserved="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021605 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021609 2560 flags.go:64] FLAG: --register-node="true" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021612 2560 flags.go:64] FLAG: --register-with-taints="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021616 2560 flags.go:64] FLAG: --registry-burst="10" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021620 2560 flags.go:64] FLAG: --registry-qps="5" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021623 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021625 2560 flags.go:64] FLAG: --reserved-memory="" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021629 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021633 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021636 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021639 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021642 2560 flags.go:64] FLAG: --runonce="false" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021645 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021648 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021651 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021654 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 01:42:49.159281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021657 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021659 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021662 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021665 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021668 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021686 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021689 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021692 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021695 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021698 2560 flags.go:64] FLAG: --system-cgroups="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021701 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021706 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021709 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021712 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021715 2560 flags.go:64] FLAG: --tls-min-version="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021717 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021720 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021726 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021728 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021731 2560 flags.go:64] FLAG: --v="2" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021740 2560 flags.go:64] FLAG: --version="false" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021743 2560 flags.go:64] FLAG: --vmodule="" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021761 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.021766 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 01:42:49.159975 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021891 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021897 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021900 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021903 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021906 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021909 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021911 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021914 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021917 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021919 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021922 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021924 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021927 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021929 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021932 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021934 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021937 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021939 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021942 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021944 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:49.160662 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021947 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021949 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021952 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021955 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021957 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021961 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021963 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021966 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021971 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021974 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021977 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021980 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021982 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021985 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021988 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021991 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021993 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021996 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.021998 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022001 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:49.161236 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022004 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022006 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022009 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022011 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022014 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022018 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022021 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022024 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022026 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022029 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022032 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022034 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022037 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022039 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022042 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022044 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022047 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022052 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022054 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:49.161794 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022057 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022060 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022063 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022066 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022068 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022071 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022074 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022076 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022079 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022082 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022084 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022087 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022089 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022092 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022096 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022099 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022102 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022105 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022107 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022110 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:49.162327 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022112 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022114 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022117 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022119 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022122 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022125 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022127 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022130 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022133 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022135 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022140 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022142 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022145 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022148 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022151 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022153 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022156 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022158 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022161 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:49.162976 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.022163 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.022754 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.030832 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.030849 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.031649 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.031656 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031732 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031738 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031741 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031744 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031747 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031750 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031753 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031756 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031759 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:49.163545 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031761 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031764 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031767 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031769 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031772 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031774 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031777 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031781 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031784 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031787 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031789 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031792 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031795 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031797 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031800 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031802 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031805 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031808 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031810 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031813 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:49.164023 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031816 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031818 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031821 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031823 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031827 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031829 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031838 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031840 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031843 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031846 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031848 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031851 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031854 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031856 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031859 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031862 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031865 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031868 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031880 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031882 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:49.164889 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031885 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031888 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031890 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031893 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031895 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031898 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031901 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031903 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031906 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031908 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031911 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031913 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031915 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031918 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031921 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031923 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031927 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031929 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031934 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031938 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:49.371285 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031942 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031945 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031948 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031954 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031957 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031960 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031963 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031966 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031969 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031971 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031974 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031977 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031980 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031982 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031985 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031987 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031990 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031993 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031995 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:49.373171 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.031999 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032002 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032005 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032008 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032011 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032014 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032018 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032021 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032024 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032026 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032029 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.032035 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032153 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032158 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032161 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 01:42:49.373725 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032164 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032167 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032171 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032174 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032177 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032180 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032183 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032186 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032189 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032193 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032196 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032199 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032202 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032204 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032207 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032209 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032212 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032214 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032217 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032220 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 01:42:49.374160 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032222 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032225 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032227 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032230 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032232 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032235 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032237 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032240 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032243 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032246 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032248 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032251 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032253 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032256 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032259 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032262 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032265 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032267 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032269 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032272 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 01:42:49.374709 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032275 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032277 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032280 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032282 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032285 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032288 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032290 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032293 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032295 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032298 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032301 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032304 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032308 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032311 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032315 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032318 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032321 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032324 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032327 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 01:42:49.375258 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032329 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032333 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032336 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032338 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032340 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032343 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032346 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032350 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032353 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032355 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032358 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032360 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032363 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032365 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032368 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032370 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032374 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032376 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032379 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032381 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 01:42:49.375795 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032384 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032386 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032389 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032391 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032393 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032396 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032398 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032402 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032405 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032408 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032411 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032413 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032416 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032420 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032423 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032425 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 01:42:49.376348 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:47.032427 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.032432 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.032555 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.035280 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.036529 2560 server.go:1019] "Starting client certificate rotation" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.036633 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.036668 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.038869 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.041667 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.051435 2560 log.go:25] "Validated CRI v1 runtime API" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.054859 2560 log.go:25] "Validated CRI v1 image API" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.055915 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 01:42:49.376806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.060428 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8de8fa48-b08c-4b08-ace5-5f858fcacbf8:/dev/nvme0n1p3 cbe7f703-b2d4-4638-9e33-d63aa8c7e0f0:/dev/nvme0n1p4] Jul 08 01:42:49.377205 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.060444 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 01:42:49.377205 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.066434 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.066771 2560 manager.go:216] Machine: {Timestamp:2026-07-08 01:42:47.065576057 +0000 UTC m=+0.281290518 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100302 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21e94e579f90732e59a6637001ff6d SystemUUID:ec21e94e-579f-9073-2e59-a6637001ff6d BootID:0884ce9f-c2ab-4f95-9f8c-5ca0dc21903a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:ba:82:d6:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:ba:82:d6:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:ff:3c:ce:ed:e2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.066882 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.066966 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.069273 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.069298 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-194.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.069968 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.069977 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.069985 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.070580 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.071772 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077634 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077779 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077809 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077818 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077831 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077841 2560 kubelet.go:398] "Adding apiserver pod source" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.077850 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 01:42:49.377384 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.078169 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.078184 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.080056 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.082089 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.082135 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083380 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083396 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083404 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083413 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083425 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083433 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083441 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083446 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083457 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083463 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083475 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.083481 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.084041 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.084049 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.087573 2560 server.go:1268] "Started kubelet" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.087773 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.087933 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.088440 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmz7j" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.088811 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.090288 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-194.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.091023 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-194.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.091187 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.091356 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.091024 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.095448 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmz7j" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.096326 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.096347 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.095738 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-194.ec2.internal.18c02d08870d1a59 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-194.ec2.internal,UID:ip-10-0-136-194.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-194.ec2.internal,},FirstTimestamp:2026-07-08 01:42:47.087487577 +0000 UTC m=+0.303202038,LastTimestamp:2026-07-08 01:42:47.087487577 +0000 UTC m=+0.303202038,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-194.ec2.internal,}" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.096542 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097057 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097112 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097125 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097233 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097242 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097697 2560 factory.go:153] Registering CRI-O factory Jul 08 01:42:49.377955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097726 2560 factory.go:223] Registration of the crio container factory successfully Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097786 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097796 2560 factory.go:55] Registering systemd factory Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097803 2560 factory.go:223] Registration of the systemd container factory successfully Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097828 2560 factory.go:103] Registering Raw factory Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.097842 2560 manager.go:1196] Started watching for new ooms in manager Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.098343 2560 manager.go:349] Starting recovery of all containers Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.098962 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.100610 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-194.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.100800 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.117688 2560 manager.go:354] Recovery completed Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.122317 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124171 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124213 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124229 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124765 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124776 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124791 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.124794 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125523 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125554 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125564 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125774 2560 policy_none.go:49] "None policy: Start" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125788 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.125797 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.126687 2560 policy_none.go:47] "Start" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163280 2560 manager.go:337] "Starting Device Plugin manager" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.163376 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163389 2560 server.go:87] "Starting device plugin registration server" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163636 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163646 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163766 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163858 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.163867 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.164375 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.164425 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.260440 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.261930 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.261970 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.261997 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.262037 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.264437 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.265212 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.265244 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.265260 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.265297 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.379047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.265960 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:47.273793 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.273813 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-194.ec2.internal\": node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:47.289446 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.123311 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal"] Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.123386 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.127324 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.127352 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.127364 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.128598 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.128696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.128722 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129431 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129455 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129465 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129431 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129531 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.129545 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.130747 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.130773 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.131385 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientMemory" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.131404 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.131415 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeHasSufficientPID" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.150950 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.159297 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.159322 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.164715 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-194.ec2.internal\" not found" node="ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.169622 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-194.ec2.internal\" not found" node="ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.252054 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.259410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.380273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.259437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1387eef4cf6fa2f7fb83b7ff4442348-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"d1387eef4cf6fa2f7fb83b7ff4442348\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.259455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.259488 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.259513 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e4a2f8f52494ec71af29c76e2e5a14-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal\" (UID: \"c2e4a2f8f52494ec71af29c76e2e5a14\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.352886 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.360233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1387eef4cf6fa2f7fb83b7ff4442348-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"d1387eef4cf6fa2f7fb83b7ff4442348\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.381136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.360282 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1387eef4cf6fa2f7fb83b7ff4442348-config\") pod \"kube-apiserver-proxy-ip-10-0-136-194.ec2.internal\" (UID: \"d1387eef4cf6fa2f7fb83b7ff4442348\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.577433 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.453406 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.577433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.467873 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.577433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.473116 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:49.577433 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.553917 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.735706 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.654536 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.755051 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.797661 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.797854 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.855297 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.862473 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 01:37:48 +0000 UTC" deadline="2028-01-14 19:42:13.937417719 +0000 UTC" Jul 08 01:42:49.924498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.862512 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13337h59m24.074909935s" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:49.956249 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-194.ec2.internal\" not found" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.963561 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:49.983385 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1387eef4cf6fa2f7fb83b7ff4442348.slice/crio-f7819517c67b4bf62b8303ca33df7dfe8a67b9593e2e4ce208941227abbc1b62 WatchSource:0}: Error finding container f7819517c67b4bf62b8303ca33df7dfe8a67b9593e2e4ce208941227abbc1b62: Status 404 returned error can't find the container with id f7819517c67b4bf62b8303ca33df7dfe8a67b9593e2e4ce208941227abbc1b62 Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:49.983687 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e4a2f8f52494ec71af29c76e2e5a14.slice/crio-bbe1666187c9463f5a6c8f3d6ff19fb96d9830eec67cda1f9e81b6844c08e812 WatchSource:0}: Error finding container bbe1666187c9463f5a6c8f3d6ff19fb96d9830eec67cda1f9e81b6844c08e812: Status 404 returned error can't find the container with id bbe1666187c9463f5a6c8f3d6ff19fb96d9830eec67cda1f9e81b6844c08e812 Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.985977 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:49.992564 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.026765 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" event={"ID":"d1387eef4cf6fa2f7fb83b7ff4442348","Type":"ContainerStarted","Data":"f7819517c67b4bf62b8303ca33df7dfe8a67b9593e2e4ce208941227abbc1b62"} Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.027663 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"c2e4a2f8f52494ec71af29c76e2e5a14","Type":"ContainerStarted","Data":"bbe1666187c9463f5a6c8f3d6ff19fb96d9830eec67cda1f9e81b6844c08e812"} Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.051279 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.057773 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.071976 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.072930 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" Jul 08 01:42:50.121027 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.081848 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 01:42:50.846427 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.840295 2560 apiserver.go:52] "Watching apiserver" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.852265 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.854003 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-x4wnz","kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc","openshift-image-registry/node-ca-sptm4","openshift-multus/multus-additional-cni-plugins-62ft8","openshift-multus/network-metrics-daemon-84kdl","openshift-network-diagnostics/network-check-target-tsjfh","openshift-network-operator/iptables-alerter-h67tv","kube-system/global-pull-secret-syncer-dscg2","openshift-cluster-node-tuning-operator/tuned-94rz4","openshift-dns/node-resolver-nx92p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal","openshift-multus/multus-vmcdq","openshift-ovn-kubernetes/ovnkube-node-7lsjc"] Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.855564 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.855640 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.857098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.858331 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.858629 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.860150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.865802 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.865897 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866187 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866188 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2vx2f\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866402 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866450 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866623 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866648 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866801 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tn5ph\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qdd88\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.866982 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-bin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-multus\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cni-binary-copy\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867563 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-multus-certs\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.922515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b727a716-41ca-45f3-8d69-50d2f5c98e57-serviceca\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4pf7\" (UniqueName: \"kubernetes.io/projected/b727a716-41ca-45f3-8d69-50d2f5c98e57-kube-api-access-z4pf7\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867635 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b727a716-41ca-45f3-8d69-50d2f5c98e57-host\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-conf-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-daemon-config\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7mp9\" (UniqueName: \"kubernetes.io/projected/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-kube-api-access-g7mp9\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-socket-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-os-release\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-hostroot\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4zmj\" (UniqueName: \"kubernetes.io/projected/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kube-api-access-z4zmj\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cnibin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.867981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-registration-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-system-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868058 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-socket-dir-parent\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.924380 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-kubelet\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868118 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-sys-fs\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-k8s-cni-cncf-io\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868240 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-netns\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-device-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.868288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-etc-kubernetes\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.870201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.872278 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.872353 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.874785 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.876057 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.876240 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.877726 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kt24m\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.878054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.879263 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.880102 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.880742 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.881154 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.881462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jqkt6\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.881712 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.882682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.883417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.883780 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.883988 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.884187 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.884225 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nxlwx\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.884361 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 01:42:50.928541 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.884676 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885092 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885552 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885611 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885866 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885866 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.885891 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-46m2x\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.886004 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vz4kd\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.886032 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.888281 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.888452 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-twchd\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.889122 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.889347 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.889597 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.890387 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.919538 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-llcsw" Jul 08 01:42:50.930689 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.926714 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-llcsw" Jul 08 01:42:50.959232 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.959206 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-kubelet\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968701 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-systemd-units\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-etc-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968797 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-kubelet\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-conf\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-k8s-cni-cncf-io\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968879 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-netns\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-device-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968926 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-netns\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968918 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-k8s-cni-cncf-io\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.968973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-device-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-kubelet-config\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:50.969041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969053 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-var-lib-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-os-release\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-netns\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-multus\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-cnibin\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-dbus\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-netd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1d94f02-e63b-4528-a166-41f84f70bf87-konnectivity-ca\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-log-socket\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b727a716-41ca-45f3-8d69-50d2f5c98e57-host\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969585 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-multus\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969615 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b727a716-41ca-45f3-8d69-50d2f5c98e57-host\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969636 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-daemon-config\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.969818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969701 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7mp9\" (UniqueName: \"kubernetes.io/projected/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-kube-api-access-g7mp9\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.969731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4zmj\" (UniqueName: \"kubernetes.io/projected/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kube-api-access-z4zmj\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970045 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336e5964-508b-486a-9621-890b19b1a73e-host-slash\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-modprobe-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-os-release\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970260 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-slash\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-env-overrides\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-sys\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-system-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-socket-dir-parent\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-sys-fs\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz7x6\" (UniqueName: \"kubernetes.io/projected/336e5964-508b-486a-9621-890b19b1a73e-kube-api-access-dz7x6\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1d94f02-e63b-4528-a166-41f84f70bf87-agent-certs\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970407 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-node-log\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd3d873b-bfa1-427f-9762-a55552177a36-ovn-node-metrics-cert\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.970648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x45lw\" (UniqueName: \"kubernetes.io/projected/cd3d873b-bfa1-427f-9762-a55552177a36-kube-api-access-x45lw\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-kubelet\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-config\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysconfig\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-run\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970683 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-os-release\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970742 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-host\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970768 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36edce4a-98a9-400a-89aa-2f7f731ffda4-hosts-file\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-etc-kubernetes\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m42sr\" (UniqueName: \"kubernetes.io/projected/2ce2ec86-6512-43a7-8d3f-37472143acec-kube-api-access-m42sr\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36edce4a-98a9-400a-89aa-2f7f731ffda4-tmp-dir\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-bin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.970978 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-daemon-config\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971003 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-system-cni-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-systemd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.971472 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-socket-dir-parent\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971073 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-sys-fs\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-etc-kubernetes\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-var-lib-kubelet\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971156 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-var-lib-cni-bin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971184 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rs9\" (UniqueName: \"kubernetes.io/projected/36edce4a-98a9-400a-89aa-2f7f731ffda4-kube-api-access-t5rs9\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cni-binary-copy\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971330 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-multus-certs\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b727a716-41ca-45f3-8d69-50d2f5c98e57-serviceca\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971447 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-host-run-multus-certs\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4pf7\" (UniqueName: \"kubernetes.io/projected/b727a716-41ca-45f3-8d69-50d2f5c98e57-kube-api-access-z4pf7\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-ovn\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-conf-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-socket-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-system-cni-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-script-lib\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.974009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-kubernetes\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972320 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-systemd\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972345 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-binary-copy\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw5br\" (UniqueName: \"kubernetes.io/projected/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-kube-api-access-rw5br\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.971890 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cni-binary-copy\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972451 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-multus-conf-dir\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972396 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-hostroot\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336e5964-508b-486a-9621-890b19b1a73e-iptables-alerter-script\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.972547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-socket-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973254 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-hostroot\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973343 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b727a716-41ca-45f3-8d69-50d2f5c98e57-serviceca\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-bin\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-tuned\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cnibin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.974706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-registration-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.975454 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973656 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-cnibin\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.975454 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-lib-modules\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.975454 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973699 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dedd1c90-f3e6-4d24-9410-9609cb5380a9-registration-dir\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.975454 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-tmp\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.975454 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.973766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqsc7\" (UniqueName: \"kubernetes.io/projected/f3501467-5f35-4683-a65b-8bb4b2c856f8-kube-api-access-nqsc7\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:50.981318 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.981192 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 01:42:50.985670 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.984922 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:42:50.985670 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.984947 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:42:50.985670 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.984961 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:50.985670 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:50.985047 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:51.485015837 +0000 UTC m=+2.939571481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:50.986016 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.985927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7mp9\" (UniqueName: \"kubernetes.io/projected/74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a-kube-api-access-g7mp9\") pod \"multus-vmcdq\" (UID: \"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a\") " pod="openshift-multus/multus-vmcdq" Jul 08 01:42:50.986188 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.986162 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4zmj\" (UniqueName: \"kubernetes.io/projected/dedd1c90-f3e6-4d24-9410-9609cb5380a9-kube-api-access-z4zmj\") pod \"aws-ebs-csi-driver-node-sd9pc\" (UID: \"dedd1c90-f3e6-4d24-9410-9609cb5380a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:50.987818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:50.987787 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4pf7\" (UniqueName: \"kubernetes.io/projected/b727a716-41ca-45f3-8d69-50d2f5c98e57-kube-api-access-z4pf7\") pod \"node-ca-sptm4\" (UID: \"b727a716-41ca-45f3-8d69-50d2f5c98e57\") " pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:51.074720 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336e5964-508b-486a-9621-890b19b1a73e-iptables-alerter-script\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-bin\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074791 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-tuned\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-lib-modules\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074864 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-tmp\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074878 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-bin\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqsc7\" (UniqueName: \"kubernetes.io/projected/f3501467-5f35-4683-a65b-8bb4b2c856f8-kube-api-access-nqsc7\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.074901 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074892 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-systemd-units\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-etc-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.074983 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-lib-modules\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075027 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-conf\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075029 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-systemd-units\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075056 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075070 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-etc-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075088 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-kubelet-config\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-var-lib-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-os-release\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-netns\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075186 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-conf\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075238 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.075292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-cnibin\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-dbus\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-netd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075373 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-kubelet-config\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1d94f02-e63b-4528-a166-41f84f70bf87-konnectivity-ca\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075421 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336e5964-508b-486a-9621-890b19b1a73e-iptables-alerter-script\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075425 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-log-socket\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075462 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-var-lib-openvswitch\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075484 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-log-socket\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075493 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336e5964-508b-486a-9621-890b19b1a73e-host-slash\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075507 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-cnibin\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075524 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-modprobe-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075530 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e5523b0-6b1c-447e-b12c-3b1170259168-dbus\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075570 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-os-release\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.076137 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-slash\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075591 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-run-netns\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-env-overrides\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075611 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-cni-netd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-sys\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075614 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysctl-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075653 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336e5964-508b-486a-9621-890b19b1a73e-host-slash\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075685 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-slash\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.075465 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075735 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-modprobe-d\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075754 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.075773 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:51.575753509 +0000 UTC m=+3.030309149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075815 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-sys\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz7x6\" (UniqueName: \"kubernetes.io/projected/336e5964-508b-486a-9621-890b19b1a73e-kube-api-access-dz7x6\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1d94f02-e63b-4528-a166-41f84f70bf87-agent-certs\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075943 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-node-log\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd3d873b-bfa1-427f-9762-a55552177a36-ovn-node-metrics-cert\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.077009 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.075992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x45lw\" (UniqueName: \"kubernetes.io/projected/cd3d873b-bfa1-427f-9762-a55552177a36-kube-api-access-x45lw\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-kubelet\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-config\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysconfig\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-run\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-host\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36edce4a-98a9-400a-89aa-2f7f731ffda4-hosts-file\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m42sr\" (UniqueName: \"kubernetes.io/projected/2ce2ec86-6512-43a7-8d3f-37472143acec-kube-api-access-m42sr\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076226 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-node-log\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076232 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-env-overrides\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076264 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.076295 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36edce4a-98a9-400a-89aa-2f7f731ffda4-tmp-dir\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-systemd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.076339 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:51.576323818 +0000 UTC m=+3.030879443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:51.078152 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-var-lib-kubelet\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076360 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-systemd\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rs9\" (UniqueName: \"kubernetes.io/projected/36edce4a-98a9-400a-89aa-2f7f731ffda4-kube-api-access-t5rs9\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-ovn\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-system-cni-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076511 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-script-lib\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-kubernetes\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-systemd\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-binary-copy\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5br\" (UniqueName: \"kubernetes.io/projected/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-kube-api-access-rw5br\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076619 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36edce4a-98a9-400a-89aa-2f7f731ffda4-tmp-dir\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076713 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/36edce4a-98a9-400a-89aa-2f7f731ffda4-hosts-file\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076759 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-sysconfig\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076801 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-run\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076821 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076171 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1d94f02-e63b-4528-a166-41f84f70bf87-konnectivity-ca\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076862 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-host\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.078673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076903 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ce2ec86-6512-43a7-8d3f-37472143acec-system-cni-dir\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076511 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-host-kubelet\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.076979 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-kubernetes\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077027 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-systemd\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077125 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3501467-5f35-4683-a65b-8bb4b2c856f8-var-lib-kubelet\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077173 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cd3d873b-bfa1-427f-9762-a55552177a36-run-ovn\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077394 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-config\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077540 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ce2ec86-6512-43a7-8d3f-37472143acec-cni-binary-copy\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077557 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-tmp\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.077625 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cd3d873b-bfa1-427f-9762-a55552177a36-ovnkube-script-lib\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.078927 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1d94f02-e63b-4528-a166-41f84f70bf87-agent-certs\") pod \"konnectivity-agent-x4wnz\" (UID: \"f1d94f02-e63b-4528-a166-41f84f70bf87\") " pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.078935 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3501467-5f35-4683-a65b-8bb4b2c856f8-etc-tuned\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.079345 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.079187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd3d873b-bfa1-427f-9762-a55552177a36-ovn-node-metrics-cert\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.089120 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.089062 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqsc7\" (UniqueName: \"kubernetes.io/projected/f3501467-5f35-4683-a65b-8bb4b2c856f8-kube-api-access-nqsc7\") pod \"tuned-94rz4\" (UID: \"f3501467-5f35-4683-a65b-8bb4b2c856f8\") " pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.090209 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.090171 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m42sr\" (UniqueName: \"kubernetes.io/projected/2ce2ec86-6512-43a7-8d3f-37472143acec-kube-api-access-m42sr\") pod \"multus-additional-cni-plugins-62ft8\" (UID: \"2ce2ec86-6512-43a7-8d3f-37472143acec\") " pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.090628 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.090608 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5br\" (UniqueName: \"kubernetes.io/projected/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-kube-api-access-rw5br\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:51.090717 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.090702 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x45lw\" (UniqueName: \"kubernetes.io/projected/cd3d873b-bfa1-427f-9762-a55552177a36-kube-api-access-x45lw\") pod \"ovnkube-node-7lsjc\" (UID: \"cd3d873b-bfa1-427f-9762-a55552177a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.091107 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.091088 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz7x6\" (UniqueName: \"kubernetes.io/projected/336e5964-508b-486a-9621-890b19b1a73e-kube-api-access-dz7x6\") pod \"iptables-alerter-h67tv\" (UID: \"336e5964-508b-486a-9621-890b19b1a73e\") " pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.091193 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.091180 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rs9\" (UniqueName: \"kubernetes.io/projected/36edce4a-98a9-400a-89aa-2f7f731ffda4-kube-api-access-t5rs9\") pod \"node-resolver-nx92p\" (UID: \"36edce4a-98a9-400a-89aa-2f7f731ffda4\") " pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.171314 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.171274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vmcdq" Jul 08 01:42:51.185614 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.185581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" Jul 08 01:42:51.194423 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.194399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sptm4" Jul 08 01:42:51.203093 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.203063 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62ft8" Jul 08 01:42:51.257780 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.257740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:42:51.266782 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.266757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:42:51.278594 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.278561 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94rz4" Jul 08 01:42:51.286303 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.286280 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nx92p" Jul 08 01:42:51.302071 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.301979 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h67tv" Jul 08 01:42:51.581156 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.581115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:51.581333 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.581167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:51.581333 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.581208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:51.581333 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581320 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581357 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581379 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581387 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:52.581367919 +0000 UTC m=+4.035923552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581394 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581323 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581448 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:52.581432634 +0000 UTC m=+4.035988272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:51.581490 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:51.581465 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:52.581458644 +0000 UTC m=+4.036014267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:51.929413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.929297 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 01:37:50 +0000 UTC" deadline="2028-02-03 02:14:26.515074801 +0000 UTC" Jul 08 01:42:51.929413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:51.929353 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13800h31m34.585724545s" Jul 08 01:42:52.024116 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.024083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:52.024282 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.024230 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:42:52.101276 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.101235 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd3d873b_bfa1_427f_9762_a55552177a36.slice/crio-47f2282f0c5d14dfe633897145199a87d29281bd1120cdd6b1ccfe208893fcff WatchSource:0}: Error finding container 47f2282f0c5d14dfe633897145199a87d29281bd1120cdd6b1ccfe208893fcff: Status 404 returned error can't find the container with id 47f2282f0c5d14dfe633897145199a87d29281bd1120cdd6b1ccfe208893fcff Jul 08 01:42:52.103196 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.103106 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a03d78_5b1c_4abb_ab8b_d10f3c2ece0a.slice/crio-8e3c8423e8cb36b8960348cace66eaa33dbfbee7dd23240e9e3f36ee446357fe WatchSource:0}: Error finding container 8e3c8423e8cb36b8960348cace66eaa33dbfbee7dd23240e9e3f36ee446357fe: Status 404 returned error can't find the container with id 8e3c8423e8cb36b8960348cace66eaa33dbfbee7dd23240e9e3f36ee446357fe Jul 08 01:42:52.106243 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.106220 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce2ec86_6512_43a7_8d3f_37472143acec.slice/crio-4a3daa91ec4165739b7c053918ec41fa4a54df4deae4c8968e6d484ebebc7340 WatchSource:0}: Error finding container 4a3daa91ec4165739b7c053918ec41fa4a54df4deae4c8968e6d484ebebc7340: Status 404 returned error can't find the container with id 4a3daa91ec4165739b7c053918ec41fa4a54df4deae4c8968e6d484ebebc7340 Jul 08 01:42:52.107750 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.107726 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336e5964_508b_486a_9621_890b19b1a73e.slice/crio-1a346bb7ec0b345a7ef8919e637eda4dd9aa0356db5aa80f7d8eadf5ec3540d0 WatchSource:0}: Error finding container 1a346bb7ec0b345a7ef8919e637eda4dd9aa0356db5aa80f7d8eadf5ec3540d0: Status 404 returned error can't find the container with id 1a346bb7ec0b345a7ef8919e637eda4dd9aa0356db5aa80f7d8eadf5ec3540d0 Jul 08 01:42:52.108167 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.108147 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d94f02_e63b_4528_a166_41f84f70bf87.slice/crio-c80e3db9a75b5ef8dca3d01dc1623ccbfe2764d4536fec7857574c6cb319d88b WatchSource:0}: Error finding container c80e3db9a75b5ef8dca3d01dc1623ccbfe2764d4536fec7857574c6cb319d88b: Status 404 returned error can't find the container with id c80e3db9a75b5ef8dca3d01dc1623ccbfe2764d4536fec7857574c6cb319d88b Jul 08 01:42:52.109500 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.109478 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36edce4a_98a9_400a_89aa_2f7f731ffda4.slice/crio-cbc7882b080c4db2548899a55e1f6f2d667c393f1c9d7605edda15e1c547b875 WatchSource:0}: Error finding container cbc7882b080c4db2548899a55e1f6f2d667c393f1c9d7605edda15e1c547b875: Status 404 returned error can't find the container with id cbc7882b080c4db2548899a55e1f6f2d667c393f1c9d7605edda15e1c547b875 Jul 08 01:42:52.110648 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.110625 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedd1c90_f3e6_4d24_9410_9609cb5380a9.slice/crio-7f7425a0c4db450c685bd05a6014086f8baeae5856d6657824a6a505bd6db9b0 WatchSource:0}: Error finding container 7f7425a0c4db450c685bd05a6014086f8baeae5856d6657824a6a505bd6db9b0: Status 404 returned error can't find the container with id 7f7425a0c4db450c685bd05a6014086f8baeae5856d6657824a6a505bd6db9b0 Jul 08 01:42:52.111550 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.111459 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb727a716_41ca_45f3_8d69_50d2f5c98e57.slice/crio-186d3a9074df23e03abf9528dc7b87703540890c40885b1d048134e614eb579b WatchSource:0}: Error finding container 186d3a9074df23e03abf9528dc7b87703540890c40885b1d048134e614eb579b: Status 404 returned error can't find the container with id 186d3a9074df23e03abf9528dc7b87703540890c40885b1d048134e614eb579b Jul 08 01:42:52.112933 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:42:52.112911 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3501467_5f35_4683_a65b_8bb4b2c856f8.slice/crio-65461b03c9a138462e0a670dbbeb41c0d3b0c9286e4d9355bf0d36dae9f1ea36 WatchSource:0}: Error finding container 65461b03c9a138462e0a670dbbeb41c0d3b0c9286e4d9355bf0d36dae9f1ea36: Status 404 returned error can't find the container with id 65461b03c9a138462e0a670dbbeb41c0d3b0c9286e4d9355bf0d36dae9f1ea36 Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.587898 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588157 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.588208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588237 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:54.588214147 +0000 UTC m=+6.042769782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.588294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588332 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588353 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588366 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588400 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588426 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:54.588411918 +0000 UTC m=+6.042967558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:52.588466 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:52.588447 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:54.588437004 +0000 UTC m=+6.042992634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:52.930061 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.929973 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 01:37:50 +0000 UTC" deadline="2027-12-08 05:38:48.840105457 +0000 UTC" Jul 08 01:42:52.930061 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:52.930010 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12435h55m55.910100175s" Jul 08 01:42:53.026740 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.026519 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:53.026740 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:53.026650 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:42:53.026740 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.026729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:53.027015 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:53.026806 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:42:53.052699 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.052213 2560 generic.go:356] "Generic (PLEG): container finished" podID="c2e4a2f8f52494ec71af29c76e2e5a14" containerID="3364fde1564cee7849202740a52ee07c85a040635b0b1a6800b1307883090ed6" exitCode=0 Jul 08 01:42:53.052699 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.052357 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"c2e4a2f8f52494ec71af29c76e2e5a14","Type":"ContainerDied","Data":"3364fde1564cee7849202740a52ee07c85a040635b0b1a6800b1307883090ed6"} Jul 08 01:42:53.067410 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.067370 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94rz4" event={"ID":"f3501467-5f35-4683-a65b-8bb4b2c856f8","Type":"ContainerStarted","Data":"65461b03c9a138462e0a670dbbeb41c0d3b0c9286e4d9355bf0d36dae9f1ea36"} Jul 08 01:42:53.103492 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.103451 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h67tv" event={"ID":"336e5964-508b-486a-9621-890b19b1a73e","Type":"ContainerStarted","Data":"1a346bb7ec0b345a7ef8919e637eda4dd9aa0356db5aa80f7d8eadf5ec3540d0"} Jul 08 01:42:53.112128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.110180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sptm4" event={"ID":"b727a716-41ca-45f3-8d69-50d2f5c98e57","Type":"ContainerStarted","Data":"186d3a9074df23e03abf9528dc7b87703540890c40885b1d048134e614eb579b"} Jul 08 01:42:53.118334 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.118234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" event={"ID":"dedd1c90-f3e6-4d24-9410-9609cb5380a9","Type":"ContainerStarted","Data":"7f7425a0c4db450c685bd05a6014086f8baeae5856d6657824a6a505bd6db9b0"} Jul 08 01:42:53.129149 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.129047 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nx92p" event={"ID":"36edce4a-98a9-400a-89aa-2f7f731ffda4","Type":"ContainerStarted","Data":"cbc7882b080c4db2548899a55e1f6f2d667c393f1c9d7605edda15e1c547b875"} Jul 08 01:42:53.145875 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.145767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" event={"ID":"d1387eef4cf6fa2f7fb83b7ff4442348","Type":"ContainerStarted","Data":"a7e35f0161b494fe59419f37376b3ad12e66992d095ddf9888425162c8cd14fd"} Jul 08 01:42:53.162881 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.162804 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-194.ec2.internal" podStartSLOduration=3.162780408 podStartE2EDuration="3.162780408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:42:53.161720239 +0000 UTC m=+4.616275885" watchObservedRunningTime="2026-07-08 01:42:53.162780408 +0000 UTC m=+4.617336054" Jul 08 01:42:53.164948 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.164913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4wnz" event={"ID":"f1d94f02-e63b-4528-a166-41f84f70bf87","Type":"ContainerStarted","Data":"c80e3db9a75b5ef8dca3d01dc1623ccbfe2764d4536fec7857574c6cb319d88b"} Jul 08 01:42:53.196256 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.196166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerStarted","Data":"4a3daa91ec4165739b7c053918ec41fa4a54df4deae4c8968e6d484ebebc7340"} Jul 08 01:42:53.206799 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.206759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vmcdq" event={"ID":"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a","Type":"ContainerStarted","Data":"8e3c8423e8cb36b8960348cace66eaa33dbfbee7dd23240e9e3f36ee446357fe"} Jul 08 01:42:53.214400 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:53.214279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"47f2282f0c5d14dfe633897145199a87d29281bd1120cdd6b1ccfe208893fcff"} Jul 08 01:42:54.024383 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:54.024349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:54.024880 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.024496 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:42:54.232957 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:54.232888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" event={"ID":"c2e4a2f8f52494ec71af29c76e2e5a14","Type":"ContainerStarted","Data":"f35e0478c32a1221e95f2dcbaa910809c1702395bcda322485f0d0fcf3bf15f5"} Jul 08 01:42:54.608994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:54.608755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:54.608994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:54.608818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:54.608994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:54.608875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:54.609296 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609015 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:54.609296 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609086 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:58.609068802 +0000 UTC m=+10.063624439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609472 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609497 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609511 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609558 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:58.609542863 +0000 UTC m=+10.064098485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609558 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:54.609638 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:54.609597 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:42:58.609588095 +0000 UTC m=+10.064143732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:55.029909 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:55.029819 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:55.030336 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:55.029957 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:42:55.030336 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:55.030325 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:55.030450 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:55.030419 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:42:56.024168 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:56.023592 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:56.024168 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:56.023746 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:42:57.024008 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:57.023969 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:57.024419 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:57.024150 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:42:57.024605 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:57.024589 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:57.024715 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:57.024691 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:42:58.024244 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:58.024209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:58.024733 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.024360 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:58.646200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:58.646264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:58.646305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.646434 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.646491 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:06.646472479 +0000 UTC m=+18.101028117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.646880 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.646944 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:06.646929888 +0000 UTC m=+18.101485526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.647021 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.647035 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.647044 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:58.647117 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:58.647068 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:06.647060683 +0000 UTC m=+18.101616319 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:42:59.024728 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:59.024567 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:42:59.024728 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:59.024702 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:42:59.025238 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:42:59.025149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:42:59.025289 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:42:59.025242 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:00.024047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:00.024013 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:00.024274 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:00.024162 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:01.023657 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:01.023624 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:01.024128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:01.023624 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:01.024128 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:01.023763 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:01.024128 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:01.023797 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:02.023597 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:02.023562 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:02.023784 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:02.023704 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:03.023905 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:03.023871 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:03.024324 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:03.023909 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:03.024324 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:03.024026 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:03.025013 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:03.024153 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:04.024410 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:04.024365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:04.024898 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:04.024513 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:05.024124 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:05.024088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:05.024320 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:05.024224 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:05.024320 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:05.024287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:05.024440 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:05.024422 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:06.023728 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:06.023696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:06.024046 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.023850 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:06.707113 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:06.707014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:06.707113 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:06.707081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:06.707125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707191 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707221 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707265 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:22.707243506 +0000 UTC m=+34.161799130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707265 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707282 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:22.707275351 +0000 UTC m=+34.161830974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707291 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707305 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:43:06.707569 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:06.707357 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:22.707341598 +0000 UTC m=+34.161897238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:43:07.027906 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:07.027806 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:07.027906 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:07.027850 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:07.028124 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:07.027957 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:07.028124 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:07.028089 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:08.024029 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:08.023990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:08.024582 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:08.024112 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:09.024206 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:09.024170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:09.024673 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:09.024358 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:09.024673 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:09.024402 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:09.024673 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:09.024514 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:10.023583 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:10.023542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:10.023748 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:10.023671 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:11.024606 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.024279 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:11.025265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.024330 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:11.025265 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:11.024690 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:11.025265 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:11.024753 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:11.268534 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.268500 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94rz4" event={"ID":"f3501467-5f35-4683-a65b-8bb4b2c856f8","Type":"ContainerStarted","Data":"ca411644ff39be7dc2e31af1b40d77e5b998edb5fa966e0637b63849da9e8ec8"} Jul 08 01:43:11.269910 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.269876 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sptm4" event={"ID":"b727a716-41ca-45f3-8d69-50d2f5c98e57","Type":"ContainerStarted","Data":"f6b9bd9a3c5bbf97a2252e8708cee9522bd62dce0e1174dafb4563abfca0b139"} Jul 08 01:43:11.271214 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.271196 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" event={"ID":"dedd1c90-f3e6-4d24-9410-9609cb5380a9","Type":"ContainerStarted","Data":"8c4ca32936b9b513cbbf54f2e6dce517771abdf46c745ac57f7e4fab70409baf"} Jul 08 01:43:11.272381 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.272356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nx92p" event={"ID":"36edce4a-98a9-400a-89aa-2f7f731ffda4","Type":"ContainerStarted","Data":"972fe7615a623b797e423671fe0183b88002649ed8eb4d3a2d5bb10c81aa1a8b"} Jul 08 01:43:11.273749 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.273724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4wnz" event={"ID":"f1d94f02-e63b-4528-a166-41f84f70bf87","Type":"ContainerStarted","Data":"efd2e1cf66095efdc618caadb6301c330434cee88d0d71e4ebd54d52ae8a78b6"} Jul 08 01:43:11.275031 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.275011 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="51b085c0834849103e399817b1a379b5a166bd685d5c96b9b29f8a8ba99464f1" exitCode=0 Jul 08 01:43:11.275110 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.275070 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"51b085c0834849103e399817b1a379b5a166bd685d5c96b9b29f8a8ba99464f1"} Jul 08 01:43:11.276476 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.276409 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vmcdq" event={"ID":"74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a","Type":"ContainerStarted","Data":"d5354725f7e3b0d425f09d672b9ad114a6a609b2d1a89b176b70c7be196b73fe"} Jul 08 01:43:11.278862 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.278825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"5243b1f76011172771b8f1586ff249ede38400e5d8d34647c2d8417e51598760"} Jul 08 01:43:11.278951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.278894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"51a1c8eb021d1216902e52b183280a4e9695ca153f4a304a84952a2c819d16bf"} Jul 08 01:43:11.278951 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.278926 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"0e91044475a5b170fd46871e5d29030f61ab2b8006d7b50b6c24ecf99e723ecf"} Jul 08 01:43:11.279024 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.278955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"5b7f4a56413bd3de1061565d464b779cc38b1076eb7dc480a3f68bff1ba836b0"} Jul 08 01:43:11.279024 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.278982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"9d296128ffbe26f185bd539fb54f72d2f88c7d4be1cefab5a6a509f6be7b649e"} Jul 08 01:43:11.288630 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.288585 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-94rz4" podStartSLOduration=3.803919259 podStartE2EDuration="22.288572891s" totalImagesPullingTime="18.484653632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:11.288213671 +0000 UTC m=+22.742769317" watchObservedRunningTime="2026-07-08 01:43:11.288572891 +0000 UTC m=+22.743128543" Jul 08 01:43:11.288751 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.288730 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-194.ec2.internal" podStartSLOduration=21.28872477 podStartE2EDuration="21.28872477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:42:54.253541484 +0000 UTC m=+5.708097133" watchObservedRunningTime="2026-07-08 01:43:11.28872477 +0000 UTC m=+22.743280414" Jul 08 01:43:11.302748 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.302701 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nx92p" podStartSLOduration=3.836047298 podStartE2EDuration="22.302686287s" totalImagesPullingTime="18.466638989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:11.302577819 +0000 UTC m=+22.757133481" watchObservedRunningTime="2026-07-08 01:43:11.302686287 +0000 UTC m=+22.757241931" Jul 08 01:43:11.342609 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.342563 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vmcdq" podStartSLOduration=3.814131182 podStartE2EDuration="22.342549574s" totalImagesPullingTime="18.528418392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:11.342346981 +0000 UTC m=+22.796902626" watchObservedRunningTime="2026-07-08 01:43:11.342549574 +0000 UTC m=+22.797105219" Jul 08 01:43:11.357466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.357415 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sptm4" podStartSLOduration=3.892893638 podStartE2EDuration="22.357400803s" totalImagesPullingTime="18.464507165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:11.357260711 +0000 UTC m=+22.811816359" watchObservedRunningTime="2026-07-08 01:43:11.357400803 +0000 UTC m=+22.811956448" Jul 08 01:43:11.376263 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:11.376150 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x4wnz" podStartSLOduration=8.066710064 podStartE2EDuration="22.376126395s" totalImagesPullingTime="14.309416331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:11.375144501 +0000 UTC m=+22.829700146" watchObservedRunningTime="2026-07-08 01:43:11.376126395 +0000 UTC m=+22.830682041" Jul 08 01:43:12.023509 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.023473 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:12.023813 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:12.023598 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:12.284021 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.283974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h67tv" event={"ID":"336e5964-508b-486a-9621-890b19b1a73e","Type":"ContainerStarted","Data":"5e17a49af1f6e391e4a0be79ed72d73b59ec54e2fb96d7e5bc4df3657cffb693"} Jul 08 01:43:12.286897 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.286804 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" event={"ID":"dedd1c90-f3e6-4d24-9410-9609cb5380a9","Type":"ContainerStarted","Data":"c769fd76e7799437747459cd79b257ba4d3a6e49f182784f6bd63ef57c351257"} Jul 08 01:43:12.288054 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.288014 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 01:43:12.290911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.290722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"b697eda97cfa6857cb254b2a1912d2ffb1410de3d289a61fd2cd98ed2a083023"} Jul 08 01:43:12.298511 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.298451 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h67tv" podStartSLOduration=8.98907741 podStartE2EDuration="23.298433606s" totalImagesPullingTime="14.309356196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:12.29809164 +0000 UTC m=+23.752647288" watchObservedRunningTime="2026-07-08 01:43:12.298433606 +0000 UTC m=+23.752989255" Jul 08 01:43:12.943681 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.943561 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T01:43:12.288030965Z","UUID":"72da4e44-0caf-422c-80f2-2d3a666c7407","Handler":null,"Name":"","Endpoint":""} Jul 08 01:43:12.945602 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.945574 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 01:43:12.945759 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:12.945611 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 01:43:13.024414 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:13.024380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:13.024414 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:13.024408 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:13.024664 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:13.024507 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:13.024664 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:13.024612 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:13.038343 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:13.038312 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:43:13.038904 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:13.038887 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:43:14.023647 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:14.023554 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:14.024079 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:14.023688 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:14.298388 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:14.298273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"5d56d95d22882ea27254763932291370df72268ba0a321fdcd42bc9b3b94f49c"} Jul 08 01:43:14.300277 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:14.300238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" event={"ID":"dedd1c90-f3e6-4d24-9410-9609cb5380a9","Type":"ContainerStarted","Data":"dd55c1520214f608af557dd3e4d06137e7b589eaa8cedeef01b10ddd3ec55e5b"} Jul 08 01:43:14.300277 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:14.300273 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 01:43:14.317694 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:14.317634 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sd9pc" podStartSLOduration=4.193276889 podStartE2EDuration="25.317572147s" totalImagesPullingTime="21.124295258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:14.316017104 +0000 UTC m=+25.770572749" watchObservedRunningTime="2026-07-08 01:43:14.317572147 +0000 UTC m=+25.772127795" Jul 08 01:43:15.024446 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:15.024266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:15.024921 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:15.024268 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:15.024921 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:15.024541 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:15.024921 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:15.024626 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:16.023753 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:16.023667 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:16.023966 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:16.023806 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:17.023873 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.023647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:17.024509 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.023685 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:17.024509 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:17.023992 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:17.024509 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:17.024034 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:17.308665 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.308632 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="cd6cb32ec7f42503a2d63369e1f63070fcc577898865103cf57dd6ca70d70ff6" exitCode=0 Jul 08 01:43:17.308862 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.308719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"cd6cb32ec7f42503a2d63369e1f63070fcc577898865103cf57dd6ca70d70ff6"} Jul 08 01:43:17.312385 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.312356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" event={"ID":"cd3d873b-bfa1-427f-9762-a55552177a36","Type":"ContainerStarted","Data":"911c6c36f47c04d4530a8a85a784d927e9a3869d3906d544ecf1438f3963fdfa"} Jul 08 01:43:17.312606 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.312576 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:17.312694 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.312659 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:17.329891 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.328910 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:17.361337 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.361283 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" podStartSLOduration=9.796981669000001 podStartE2EDuration="28.361265999s" totalImagesPullingTime="18.56428433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:17.360960859 +0000 UTC m=+28.815516504" watchObservedRunningTime="2026-07-08 01:43:17.361265999 +0000 UTC m=+28.815821669" Jul 08 01:43:17.698183 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.698090 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:43:17.698335 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.698243 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 01:43:17.698717 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:17.698699 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x4wnz" Jul 08 01:43:18.023788 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:18.023702 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:18.023972 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:18.023857 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:18.315485 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:18.315450 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:18.330627 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:18.330603 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:19.028942 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:19.028911 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:19.029444 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:19.029036 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:19.029444 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:19.029079 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:19.029444 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:19.029200 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:19.318928 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:19.318892 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="19d04af2a02b3089146ed0047a65aa85e10b38a98397311cc19992d2064949b4" exitCode=0 Jul 08 01:43:19.319126 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:19.318985 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"19d04af2a02b3089146ed0047a65aa85e10b38a98397311cc19992d2064949b4"} Jul 08 01:43:20.023779 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:20.023745 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:20.023938 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:20.023890 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:21.023583 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:21.023547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:21.024088 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:21.023650 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:21.024088 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:21.023703 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:21.024088 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:21.023786 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:21.326248 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:21.326213 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="15d5965f21934d3a1bf2a58442db41a0f064498ace2a629e062231cca37ed2e0" exitCode=0 Jul 08 01:43:21.326396 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:21.326275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"15d5965f21934d3a1bf2a58442db41a0f064498ace2a629e062231cca37ed2e0"} Jul 08 01:43:22.024025 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:22.023990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:22.024456 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.024134 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:22.733540 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:22.733497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:22.733540 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:22.733545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:22.733573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733655 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733658 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733685 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733707 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs podName:8266ac7a-4fa1-4a13-9ca3-49137e9303d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:54.733693064 +0000 UTC m=+66.188248686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs") pod "network-metrics-daemon-84kdl" (UID: "8266ac7a-4fa1-4a13-9ca3-49137e9303d0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733708 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733720 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret podName:6e5523b0-6b1c-447e-b12c-3b1170259168 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:54.733713548 +0000 UTC m=+66.188269171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret") pod "global-pull-secret-syncer-dscg2" (UID: "6e5523b0-6b1c-447e-b12c-3b1170259168") : object "kube-system"/"original-pull-secret" not registered Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733723 2560 projected.go:196] Error preparing data for projected volume kube-api-access-kgn2m for pod openshift-network-diagnostics/network-check-target-tsjfh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:43:22.733809 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:22.733766 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m podName:ec8aef35-e3f7-40a0-b597-01b98a5deba0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:54.733751628 +0000 UTC m=+66.188307267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kgn2m" (UniqueName: "kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m") pod "network-check-target-tsjfh" (UID: "ec8aef35-e3f7-40a0-b597-01b98a5deba0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 01:43:23.027079 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:23.027012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:23.027457 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:23.027010 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:23.027457 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:23.027107 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:23.027457 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:23.027162 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:24.024053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:24.024022 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:24.024204 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:24.024133 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:25.024103 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:25.024043 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:25.024578 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:25.024174 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:25.024578 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:25.024192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:25.024578 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:25.024276 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:26.023335 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:26.023304 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:26.023526 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:26.023429 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:26.340476 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:26.340441 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="6e142b12609874a47fd2443756f58cc1cbae54034663182d6bffe638c7c76dc9" exitCode=0 Jul 08 01:43:26.340863 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:26.340506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"6e142b12609874a47fd2443756f58cc1cbae54034663182d6bffe638c7c76dc9"} Jul 08 01:43:27.024432 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:27.024401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:27.024615 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:27.024512 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:27.024615 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:27.024572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:27.024697 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:27.024662 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:27.345309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:27.345280 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ce2ec86-6512-43a7-8d3f-37472143acec" containerID="faf1e7441672f5ee427e07d75d051831804c0c72657546a36a64a0b3954a2c9b" exitCode=0 Jul 08 01:43:27.345682 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:27.345316 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerDied","Data":"faf1e7441672f5ee427e07d75d051831804c0c72657546a36a64a0b3954a2c9b"} Jul 08 01:43:28.023723 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:28.023542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:28.023948 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:28.023812 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:28.351147 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:28.351114 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62ft8" event={"ID":"2ce2ec86-6512-43a7-8d3f-37472143acec","Type":"ContainerStarted","Data":"5ad61d33f05457e94f2f3403dec475a89241aeb58231a2613d699f4f1e6f4d8d"} Jul 08 01:43:28.381418 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:28.381364 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-62ft8" podStartSLOduration=8.252401478 podStartE2EDuration="39.381349336s" totalImagesPullingTime="31.128947858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:28.380488928 +0000 UTC m=+39.835044570" watchObservedRunningTime="2026-07-08 01:43:28.381349336 +0000 UTC m=+39.835904983" Jul 08 01:43:29.024717 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.024681 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:29.024909 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:29.024801 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:29.024909 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.024855 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:29.025042 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:29.024982 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:29.066716 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.066680 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84kdl"] Jul 08 01:43:29.067041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.067022 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:29.067199 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:29.067177 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:29.069942 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.069917 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dscg2"] Jul 08 01:43:29.070772 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.070746 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsjfh"] Jul 08 01:43:29.353779 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.353746 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:29.354568 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:29.353756 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:29.354568 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:29.353867 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:29.354568 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:29.353936 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:31.023682 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:31.023649 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:31.024262 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:31.023713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:31.024262 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:31.023649 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:31.024262 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:31.023775 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dscg2" podUID="6e5523b0-6b1c-447e-b12c-3b1170259168" Jul 08 01:43:31.024262 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:31.023843 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsjfh" podUID="ec8aef35-e3f7-40a0-b597-01b98a5deba0" Jul 08 01:43:31.024262 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:31.023930 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84kdl" podUID="8266ac7a-4fa1-4a13-9ca3-49137e9303d0" Jul 08 01:43:32.338355 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.338328 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-194.ec2.internal" event="NodeReady" Jul 08 01:43:32.338819 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.338472 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 01:43:32.380424 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.380384 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7568b7fc95-rg5bd"] Jul 08 01:43:32.401692 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.401663 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7568b7fc95-rg5bd"] Jul 08 01:43:32.401692 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.401700 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f58sk"] Jul 08 01:43:32.401937 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.401765 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.405155 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405091 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 01:43:32.405155 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405091 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-65qmt\"" Jul 08 01:43:32.405409 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405390 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 01:43:32.405762 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-installation-pull-secrets\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.405798 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405775 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 01:43:32.405847 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-tls\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.405889 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-image-registry-private-configuration\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.405937 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkwr\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-kube-api-access-ktkwr\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.405987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.405949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-ca-trust-extracted\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.406035 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.406004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-trusted-ca\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.406083 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.406034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-certificates\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.406083 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.406075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-bound-sa-token\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.411915 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.411753 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 01:43:32.425826 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.425765 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7xfl5"] Jul 08 01:43:32.425977 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.425929 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.429109 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.429091 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v5bn9\"" Jul 08 01:43:32.429774 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.429758 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 01:43:32.429998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.429982 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 01:43:32.449898 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.449863 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f58sk"] Jul 08 01:43:32.449898 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.449906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7xfl5"] Jul 08 01:43:32.450079 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.449973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.452483 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.452457 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 01:43:32.452813 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.452540 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 01:43:32.453103 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.453074 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 01:43:32.453271 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.453251 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 01:43:32.453414 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.453393 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wwxml\"" Jul 08 01:43:32.496153 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.496123 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-24xpb"] Jul 08 01:43:32.506607 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-installation-pull-secrets\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.506759 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.506759 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-config-volume\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.506873 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-tmp-dir\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.506873 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-tls\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.506984 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-image-registry-private-configuration\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.506984 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.506977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkwr\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-kube-api-access-ktkwr\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.507081 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73313f24-d6a1-4d65-86b6-9a667da7efa8-crio-socket\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.507081 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-ca-trust-extracted\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.507188 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-trusted-ca\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.507188 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-certificates\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.507287 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-bound-sa-token\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.507287 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-metrics-tls\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.507287 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507260 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6bp5\" (UniqueName: \"kubernetes.io/projected/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-kube-api-access-t6bp5\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.507433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73313f24-d6a1-4d65-86b6-9a667da7efa8-data-volume\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.507433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73313f24-d6a1-4d65-86b6-9a667da7efa8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.507433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7rhp\" (UniqueName: \"kubernetes.io/projected/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-api-access-g7rhp\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.507551 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.507460 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-ca-trust-extracted\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.508208 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.508183 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-trusted-ca\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.508303 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.508228 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-certificates\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.511058 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.511031 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-registry-tls\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.511166 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.511039 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-installation-pull-secrets\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.515971 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.515952 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24xpb"] Jul 08 01:43:32.516099 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.516084 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.516167 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.516116 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-bound-sa-token\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.516871 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.516823 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkwr\" (UniqueName: \"kubernetes.io/projected/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-kube-api-access-ktkwr\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.518183 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.518164 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3-image-registry-private-configuration\") pod \"image-registry-7568b7fc95-rg5bd\" (UID: \"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3\") " pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.519057 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.519039 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 01:43:32.519057 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.519057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 01:43:32.519221 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.519057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-psvfg\"" Jul 08 01:43:32.519338 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.519323 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 01:43:32.608371 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-metrics-tls\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.608371 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608370 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6bp5\" (UniqueName: \"kubernetes.io/projected/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-kube-api-access-t6bp5\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.608624 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73313f24-d6a1-4d65-86b6-9a667da7efa8-data-volume\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.608624 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73313f24-d6a1-4d65-86b6-9a667da7efa8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.608718 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7rhp\" (UniqueName: \"kubernetes.io/projected/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-api-access-g7rhp\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.608801 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73313f24-d6a1-4d65-86b6-9a667da7efa8-data-volume\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.608801 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7zld\" (UniqueName: \"kubernetes.io/projected/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-kube-api-access-r7zld\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.608945 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608900 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.608945 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-config-volume\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.609058 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.608958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-tmp-dir\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.609058 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73313f24-d6a1-4d65-86b6-9a667da7efa8-crio-socket\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.609058 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-cert\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.609263 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609243 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73313f24-d6a1-4d65-86b6-9a667da7efa8-crio-socket\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.609352 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-tmp-dir\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.609474 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609457 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.609592 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.609481 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-config-volume\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.610815 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.610796 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-metrics-tls\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.611035 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.611018 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73313f24-d6a1-4d65-86b6-9a667da7efa8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.618743 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.618712 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7rhp\" (UniqueName: \"kubernetes.io/projected/73313f24-d6a1-4d65-86b6-9a667da7efa8-kube-api-access-g7rhp\") pod \"insights-runtime-extractor-7xfl5\" (UID: \"73313f24-d6a1-4d65-86b6-9a667da7efa8\") " pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.619811 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.619794 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6bp5\" (UniqueName: \"kubernetes.io/projected/6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58-kube-api-access-t6bp5\") pod \"dns-default-f58sk\" (UID: \"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58\") " pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.710146 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.710037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7zld\" (UniqueName: \"kubernetes.io/projected/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-kube-api-access-r7zld\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.710276 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.710176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-cert\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.711670 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.711647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:32.712513 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.712489 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-cert\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.718762 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.718744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7zld\" (UniqueName: \"kubernetes.io/projected/09e2edcb-e53d-4a92-93d1-99c86bd8ce6e-kube-api-access-r7zld\") pod \"ingress-canary-24xpb\" (UID: \"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e\") " pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.734007 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.733986 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:32.761851 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.761810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7xfl5" Jul 08 01:43:32.832774 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.829047 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24xpb" Jul 08 01:43:32.928598 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.928568 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f58sk"] Jul 08 01:43:32.932021 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.931999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7568b7fc95-rg5bd"] Jul 08 01:43:32.933042 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:32.933012 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc4451f_f41c_4c4f_a1b6_cbe0b0728e58.slice/crio-538f8c6bf7ec425f8baf58eadb18d8a61a02f4efa5172f7a89868998b30d9250 WatchSource:0}: Error finding container 538f8c6bf7ec425f8baf58eadb18d8a61a02f4efa5172f7a89868998b30d9250: Status 404 returned error can't find the container with id 538f8c6bf7ec425f8baf58eadb18d8a61a02f4efa5172f7a89868998b30d9250 Jul 08 01:43:32.936221 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:32.936200 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb725eeb9_8fd4_4e37_b9f4_38188cb7b8e3.slice/crio-67f35cd2c489c89dbf564047e35acbf567d0fa6cecbb7443ecc0f2c3f821c44f WatchSource:0}: Error finding container 67f35cd2c489c89dbf564047e35acbf567d0fa6cecbb7443ecc0f2c3f821c44f: Status 404 returned error can't find the container with id 67f35cd2c489c89dbf564047e35acbf567d0fa6cecbb7443ecc0f2c3f821c44f Jul 08 01:43:32.952828 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.952796 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7xfl5"] Jul 08 01:43:32.955605 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:32.955573 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73313f24_d6a1_4d65_86b6_9a667da7efa8.slice/crio-3dd828aac3d8f8182d3b259dd2dc3da7357fd3941d363266a99620f72cd9609d WatchSource:0}: Error finding container 3dd828aac3d8f8182d3b259dd2dc3da7357fd3941d363266a99620f72cd9609d: Status 404 returned error can't find the container with id 3dd828aac3d8f8182d3b259dd2dc3da7357fd3941d363266a99620f72cd9609d Jul 08 01:43:32.990012 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:32.989985 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24xpb"] Jul 08 01:43:32.994232 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:32.994204 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e2edcb_e53d_4a92_93d1_99c86bd8ce6e.slice/crio-a1fdf321aa52244805e2fcbae40c15385b039a246a3468c85bd4e770ed5c9b03 WatchSource:0}: Error finding container a1fdf321aa52244805e2fcbae40c15385b039a246a3468c85bd4e770ed5c9b03: Status 404 returned error can't find the container with id a1fdf321aa52244805e2fcbae40c15385b039a246a3468c85bd4e770ed5c9b03 Jul 08 01:43:33.024303 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.024276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:33.024425 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.024275 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:33.024425 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.024398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:33.027203 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.027181 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 01:43:33.027347 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.027249 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 01:43:33.027347 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.027296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2x5xt\"" Jul 08 01:43:33.028321 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.028303 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 01:43:33.028419 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.028308 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkcps\"" Jul 08 01:43:33.028419 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.028309 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 01:43:33.365585 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.365390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" event={"ID":"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3","Type":"ContainerStarted","Data":"4b84b7cb75f7b95918782adeb92ecf5f1fef60a0a2eb9df47410914529ac0bc3"} Jul 08 01:43:33.366004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.365611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" event={"ID":"b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3","Type":"ContainerStarted","Data":"67f35cd2c489c89dbf564047e35acbf567d0fa6cecbb7443ecc0f2c3f821c44f"} Jul 08 01:43:33.366004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.365637 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:33.366514 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.366484 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f58sk" event={"ID":"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58","Type":"ContainerStarted","Data":"538f8c6bf7ec425f8baf58eadb18d8a61a02f4efa5172f7a89868998b30d9250"} Jul 08 01:43:33.367569 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.367545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24xpb" event={"ID":"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e","Type":"ContainerStarted","Data":"a1fdf321aa52244805e2fcbae40c15385b039a246a3468c85bd4e770ed5c9b03"} Jul 08 01:43:33.368610 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:33.368586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xfl5" event={"ID":"73313f24-d6a1-4d65-86b6-9a667da7efa8","Type":"ContainerStarted","Data":"3dd828aac3d8f8182d3b259dd2dc3da7357fd3941d363266a99620f72cd9609d"} Jul 08 01:43:39.388104 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:39.388067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f58sk" event={"ID":"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58","Type":"ContainerStarted","Data":"1327894cb205c336d1a9ebda0dfddfbf399b34dde27b49e0ac3f6a628ca8a71d"} Jul 08 01:43:39.389680 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:39.389651 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24xpb" event={"ID":"09e2edcb-e53d-4a92-93d1-99c86bd8ce6e","Type":"ContainerStarted","Data":"3efbb78685b0fcb4d3b3a5ab14231f4db0bac4d210cd7c0afa76d8f7c4a0eb07"} Jul 08 01:43:39.406664 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:39.406572 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" podStartSLOduration=11.406554632 podStartE2EDuration="11.406554632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:33.383663752 +0000 UTC m=+44.838219399" watchObservedRunningTime="2026-07-08 01:43:39.406554632 +0000 UTC m=+50.861110277" Jul 08 01:43:39.406942 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:39.406911 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-24xpb" podStartSLOduration=1.134520193 podStartE2EDuration="7.406900586s" totalImagesPullingTime="6.272380393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:39.405508243 +0000 UTC m=+50.860063892" watchObservedRunningTime="2026-07-08 01:43:39.406900586 +0000 UTC m=+50.861456412" Jul 08 01:43:40.393409 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:40.393211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xfl5" event={"ID":"73313f24-d6a1-4d65-86b6-9a667da7efa8","Type":"ContainerStarted","Data":"45e97c25c3880eed60a5ae303342fa2b50a9de87cc4b54ac6ddfb00958af297a"} Jul 08 01:43:40.394824 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:40.394795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f58sk" event={"ID":"6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58","Type":"ContainerStarted","Data":"82a2e882bb36b9f0ff4e634d6b76d7420a2d27c80abdc94758c05240cc1224bf"} Jul 08 01:43:40.412957 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:40.412900 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f58sk" podStartSLOduration=2.085910161 podStartE2EDuration="8.412878933s" totalImagesPullingTime="6.326968772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:40.412376217 +0000 UTC m=+51.866931877" watchObservedRunningTime="2026-07-08 01:43:40.412878933 +0000 UTC m=+51.867434580" Jul 08 01:43:41.401801 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:41.401767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xfl5" event={"ID":"73313f24-d6a1-4d65-86b6-9a667da7efa8","Type":"ContainerStarted","Data":"5be7e048141b8dadfbc0c5bf3b5616714d2153289af152e35f4ab486e5235962"} Jul 08 01:43:41.402260 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:41.402036 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:43.822138 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.822105 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q"] Jul 08 01:43:43.833006 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.832967 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:43.833667 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.833640 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q"] Jul 08 01:43:43.835076 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.835058 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 01:43:43.835633 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.835508 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r2z2w\"" Jul 08 01:43:43.835633 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.835598 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 01:43:43.836553 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.836519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 01:43:43.836651 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.836561 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 01:43:43.836651 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.836604 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 01:43:43.911892 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.911857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b2f2938-4c53-4322-93c3-5fddf586aa49-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:43.911892 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.911903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:43.912142 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.911943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plsl9\" (UniqueName: \"kubernetes.io/projected/0b2f2938-4c53-4322-93c3-5fddf586aa49-kube-api-access-plsl9\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:43.912142 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:43.912015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.013150 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.013108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plsl9\" (UniqueName: \"kubernetes.io/projected/0b2f2938-4c53-4322-93c3-5fddf586aa49-kube-api-access-plsl9\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.013421 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.013163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.013421 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.013198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b2f2938-4c53-4322-93c3-5fddf586aa49-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.013421 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.013224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.013421 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:44.013321 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 01:43:44.013421 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:44.013387 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls podName:0b2f2938-4c53-4322-93c3-5fddf586aa49 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:44.513371618 +0000 UTC m=+55.967927256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-wg46q" (UID: "0b2f2938-4c53-4322-93c3-5fddf586aa49") : secret "prometheus-operator-tls" not found Jul 08 01:43:44.014013 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.013990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b2f2938-4c53-4322-93c3-5fddf586aa49-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.017482 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.017455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.021173 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.021151 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plsl9\" (UniqueName: \"kubernetes.io/projected/0b2f2938-4c53-4322-93c3-5fddf586aa49-kube-api-access-plsl9\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.414258 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.414219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7xfl5" event={"ID":"73313f24-d6a1-4d65-86b6-9a667da7efa8","Type":"ContainerStarted","Data":"8852ede8e70d879d91b2a6aae2db58be56a0df829d3d1d947e48eb6e1191f0d3"} Jul 08 01:43:44.435876 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.435803 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7xfl5" podStartSLOduration=2.123804117 podStartE2EDuration="12.435762428s" totalImagesPullingTime="10.311958311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:43:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:44.435091915 +0000 UTC m=+55.889647560" watchObservedRunningTime="2026-07-08 01:43:44.435762428 +0000 UTC m=+55.890318073" Jul 08 01:43:44.518003 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.517959 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.520537 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.520507 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b2f2938-4c53-4322-93c3-5fddf586aa49-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-wg46q\" (UID: \"0b2f2938-4c53-4322-93c3-5fddf586aa49\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.742938 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.742825 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" Jul 08 01:43:44.868362 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:44.868332 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q"] Jul 08 01:43:44.871848 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:44.871811 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b2f2938_4c53_4322_93c3_5fddf586aa49.slice/crio-90edf5d1d7758e8e6943d5de5b2a0171fb3a3a2eedcfe68d9df886c5ead46bdb WatchSource:0}: Error finding container 90edf5d1d7758e8e6943d5de5b2a0171fb3a3a2eedcfe68d9df886c5ead46bdb: Status 404 returned error can't find the container with id 90edf5d1d7758e8e6943d5de5b2a0171fb3a3a2eedcfe68d9df886c5ead46bdb Jul 08 01:43:45.422062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:45.422013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" event={"ID":"0b2f2938-4c53-4322-93c3-5fddf586aa49","Type":"ContainerStarted","Data":"90edf5d1d7758e8e6943d5de5b2a0171fb3a3a2eedcfe68d9df886c5ead46bdb"} Jul 08 01:43:46.427070 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:46.426980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" event={"ID":"0b2f2938-4c53-4322-93c3-5fddf586aa49","Type":"ContainerStarted","Data":"ab51d3e316bbea3d2be3850817e6c6fa9a78f0575fb7bbf2048e703d2a43cb7e"} Jul 08 01:43:46.427070 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:46.427040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" event={"ID":"0b2f2938-4c53-4322-93c3-5fddf586aa49","Type":"ContainerStarted","Data":"9a592be705efdea0e183d53868f801ccce1b244292106358130de5aa2d8ef835"} Jul 08 01:43:46.446036 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:46.445987 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-wg46q" podStartSLOduration=2.162724538 podStartE2EDuration="3.445971274s" totalImagesPullingTime="1.283246736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:46.444965846 +0000 UTC m=+57.899521491" watchObservedRunningTime="2026-07-08 01:43:46.445971274 +0000 UTC m=+57.900526957" Jul 08 01:43:48.209775 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.209739 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mb482"] Jul 08 01:43:48.212049 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.212027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.214662 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.214638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 01:43:48.214784 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.214639 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 01:43:48.214784 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.214707 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 01:43:48.217706 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.217687 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-658t5\"" Jul 08 01:43:48.349413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-wtmp\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-metrics-client-ca\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsps8\" (UniqueName: \"kubernetes.io/projected/2951124e-c4b7-43e0-a74e-eb932dda57d0-kube-api-access-lsps8\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-sys\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-root\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.349790 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.349743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-textfile\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450314 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-metrics-client-ca\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsps8\" (UniqueName: \"kubernetes.io/projected/2951124e-c4b7-43e0-a74e-eb932dda57d0-kube-api-access-lsps8\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-sys\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-root\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-textfile\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450443 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-sys\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-root\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450497 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450554 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-wtmp\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:48.450708 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:48.450773 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls podName:2951124e-c4b7-43e0-a74e-eb932dda57d0 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:48.950754241 +0000 UTC m=+60.405309877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls") pod "node-exporter-mb482" (UID: "2951124e-c4b7-43e0-a74e-eb932dda57d0") : secret "node-exporter-tls" not found Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450794 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-wtmp\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.450869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.450807 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-textfile\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.451170 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.451025 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-metrics-client-ca\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.451170 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.451070 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.452955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.452940 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.459110 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.459084 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsps8\" (UniqueName: \"kubernetes.io/projected/2951124e-c4b7-43e0-a74e-eb932dda57d0-kube-api-access-lsps8\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.954051 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.954011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:48.956248 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:48.956224 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2951124e-c4b7-43e0-a74e-eb932dda57d0-node-exporter-tls\") pod \"node-exporter-mb482\" (UID: \"2951124e-c4b7-43e0-a74e-eb932dda57d0\") " pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:49.123047 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.123015 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-658t5\"" Jul 08 01:43:49.130990 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.130964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mb482" Jul 08 01:43:49.141344 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:49.141310 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2951124e_c4b7_43e0_a74e_eb932dda57d0.slice/crio-90ce2acb704124f701e0ba6f974ce302f8a07260d5b10d545a15e00b463a7c6e WatchSource:0}: Error finding container 90ce2acb704124f701e0ba6f974ce302f8a07260d5b10d545a15e00b463a7c6e: Status 404 returned error can't find the container with id 90ce2acb704124f701e0ba6f974ce302f8a07260d5b10d545a15e00b463a7c6e Jul 08 01:43:49.300226 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.300136 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:43:49.327860 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.327817 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:43:49.328035 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.327971 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.331409 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.331306 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 01:43:49.332757 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332734 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 01:43:49.332987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332738 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 01:43:49.332987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332781 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 01:43:49.332987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332800 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 01:43:49.332987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332735 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 01:43:49.332987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.332984 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 01:43:49.333287 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.333270 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nz77x\"" Jul 08 01:43:49.333348 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.333288 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 01:43:49.333348 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.333315 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 01:43:49.437269 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.437231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mb482" event={"ID":"2951124e-c4b7-43e0-a74e-eb932dda57d0","Type":"ContainerStarted","Data":"90ce2acb704124f701e0ba6f974ce302f8a07260d5b10d545a15e00b463a7c6e"} Jul 08 01:43:49.456695 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.456695 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.456917 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.456917 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.456917 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7t75\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.456917 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456940 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.456961 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.457021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.457050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457197 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.457081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.457197 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.457124 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557466 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557662 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557715 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557750 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557785 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557785 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557898 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.557960 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7t75\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.557985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558217 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.558016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558217 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.558047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.558675 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.558648 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.560926 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.560749 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.560926 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.560773 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.560926 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.560867 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.561183 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.561134 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.561469 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.561443 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.561552 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.561482 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.561552 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.561503 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.561950 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.561932 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.562498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.562478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.562847 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.562817 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.565826 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.565804 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7t75\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75\") pod \"alertmanager-main-0\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.638955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.638926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:43:49.800362 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:49.800329 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:43:49.806237 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:49.806205 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c09b0b7_1a7f_45f8_9864_f77a00e3db2e.slice/crio-72c37dfdc375871a944e71341a870e854da0ebb8dc2d717e71b8b92e461a59ea WatchSource:0}: Error finding container 72c37dfdc375871a944e71341a870e854da0ebb8dc2d717e71b8b92e461a59ea: Status 404 returned error can't find the container with id 72c37dfdc375871a944e71341a870e854da0ebb8dc2d717e71b8b92e461a59ea Jul 08 01:43:50.338404 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:50.338371 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7lsjc" Jul 08 01:43:50.441792 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:50.441755 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"72c37dfdc375871a944e71341a870e854da0ebb8dc2d717e71b8b92e461a59ea"} Jul 08 01:43:51.407948 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:51.407916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f58sk" Jul 08 01:43:51.446762 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:51.446727 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7" exitCode=0 Jul 08 01:43:51.446987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:51.446887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7"} Jul 08 01:43:51.449238 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:51.449207 2560 generic.go:356] "Generic (PLEG): container finished" podID="2951124e-c4b7-43e0-a74e-eb932dda57d0" containerID="fe76be047860ca4c8a4e0f48d150a5c418544ebd49d6af8b4f6577cdcf60babf" exitCode=0 Jul 08 01:43:51.449335 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:51.449250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mb482" event={"ID":"2951124e-c4b7-43e0-a74e-eb932dda57d0","Type":"ContainerDied","Data":"fe76be047860ca4c8a4e0f48d150a5c418544ebd49d6af8b4f6577cdcf60babf"} Jul 08 01:43:52.455363 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.455324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mb482" event={"ID":"2951124e-c4b7-43e0-a74e-eb932dda57d0","Type":"ContainerStarted","Data":"8c1ef4206ca2ff5f73347b23b4cab1a06e57ada906d7e03ce959d8644f948db5"} Jul 08 01:43:52.455825 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.455392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mb482" event={"ID":"2951124e-c4b7-43e0-a74e-eb932dda57d0","Type":"ContainerStarted","Data":"36fec40187b05b0cac67519cdc17dd3ea1a0df3ccb33d8ec41ffc4471b469d86"} Jul 08 01:43:52.476681 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.476629 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mb482" podStartSLOduration=3.034511245 podStartE2EDuration="4.476611035s" totalImagesPullingTime="1.368443155s" totalInitContainerRuntime="73.656635ms" isStatefulPod=true podCreationTimestamp="2026-07-08 01:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:52.47523744 +0000 UTC m=+63.929793087" watchObservedRunningTime="2026-07-08 01:43:52.476611035 +0000 UTC m=+63.931166721" Jul 08 01:43:52.716221 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.716130 2560 patch_prober.go:28] interesting pod/image-registry-7568b7fc95-rg5bd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 01:43:52.716221 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.716188 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" podUID="b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 01:43:52.944815 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.944777 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt"] Jul 08 01:43:52.948436 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.948413 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:52.950806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.950718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-h7cfx\"" Jul 08 01:43:52.950806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.950768 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 01:43:52.954887 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:52.954862 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt"] Jul 08 01:43:53.090064 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.090022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4r9c\" (UniqueName: \"kubernetes.io/projected/8cddad98-596f-4423-b667-a980ad86c8f1-kube-api-access-n4r9c\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.090256 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.090121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.191247 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.191210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4r9c\" (UniqueName: \"kubernetes.io/projected/8cddad98-596f-4423-b667-a980ad86c8f1-kube-api-access-n4r9c\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.191469 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.191280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.191469 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:53.191393 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 01:43:53.191469 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:43:53.191464 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert podName:8cddad98-596f-4423-b667-a980ad86c8f1 nodeName:}" failed. No retries permitted until 2026-07-08 01:43:53.691444108 +0000 UTC m=+65.145999736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-n2rlt" (UID: "8cddad98-596f-4423-b667-a980ad86c8f1") : secret "monitoring-plugin-cert" not found Jul 08 01:43:53.200375 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.200347 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4r9c\" (UniqueName: \"kubernetes.io/projected/8cddad98-596f-4423-b667-a980ad86c8f1-kube-api-access-n4r9c\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.463568 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.463528 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234"} Jul 08 01:43:53.464010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.463619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e"} Jul 08 01:43:53.464010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.463664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece"} Jul 08 01:43:53.464010 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.463713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e"} Jul 08 01:43:53.695257 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.695144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.697539 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.697515 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cddad98-596f-4423-b667-a980ad86c8f1-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-n2rlt\" (UID: \"8cddad98-596f-4423-b667-a980ad86c8f1\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.861248 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.861209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:53.984654 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:53.984581 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt"] Jul 08 01:43:53.987484 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:53.987458 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cddad98_596f_4423_b667_a980ad86c8f1.slice/crio-954c297d5e54e17f7fae8b6f04ee106c5154697c988461fecdfde0a8f5fd8de4 WatchSource:0}: Error finding container 954c297d5e54e17f7fae8b6f04ee106c5154697c988461fecdfde0a8f5fd8de4: Status 404 returned error can't find the container with id 954c297d5e54e17f7fae8b6f04ee106c5154697c988461fecdfde0a8f5fd8de4 Jul 08 01:43:54.378751 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.378723 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7568b7fc95-rg5bd" Jul 08 01:43:54.469235 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.469201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" event={"ID":"8cddad98-596f-4423-b667-a980ad86c8f1","Type":"ContainerStarted","Data":"954c297d5e54e17f7fae8b6f04ee106c5154697c988461fecdfde0a8f5fd8de4"} Jul 08 01:43:54.473085 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.473058 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf"} Jul 08 01:43:54.805599 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.805073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:54.805599 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.805139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:54.805599 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.805183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:54.807911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.807877 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 01:43:54.808056 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.807911 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 01:43:54.808056 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.807913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 01:43:54.818998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.818967 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8266ac7a-4fa1-4a13-9ca3-49137e9303d0-metrics-certs\") pod \"network-metrics-daemon-84kdl\" (UID: \"8266ac7a-4fa1-4a13-9ca3-49137e9303d0\") " pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:54.818998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.818993 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 01:43:54.819511 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.819486 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e5523b0-6b1c-447e-b12c-3b1170259168-original-pull-secret\") pod \"global-pull-secret-syncer-dscg2\" (UID: \"6e5523b0-6b1c-447e-b12c-3b1170259168\") " pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:54.829952 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.829919 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgn2m\" (UniqueName: \"kubernetes.io/projected/ec8aef35-e3f7-40a0-b597-01b98a5deba0-kube-api-access-kgn2m\") pod \"network-check-target-tsjfh\" (UID: \"ec8aef35-e3f7-40a0-b597-01b98a5deba0\") " pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:54.936256 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.936220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2x5xt\"" Jul 08 01:43:54.942253 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.942219 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pkcps\"" Jul 08 01:43:54.944925 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.944905 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:43:54.945035 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.944915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dscg2" Jul 08 01:43:54.950946 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:54.950924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84kdl" Jul 08 01:43:55.131180 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.131151 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dscg2"] Jul 08 01:43:55.148731 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.148701 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsjfh"] Jul 08 01:43:55.151628 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.151597 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84kdl"] Jul 08 01:43:55.461888 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:55.461798 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e5523b0_6b1c_447e_b12c_3b1170259168.slice/crio-536922b33074b70f27216ae20a15d71f110ddea2beff6b98f6b1a719a944ad1c WatchSource:0}: Error finding container 536922b33074b70f27216ae20a15d71f110ddea2beff6b98f6b1a719a944ad1c: Status 404 returned error can't find the container with id 536922b33074b70f27216ae20a15d71f110ddea2beff6b98f6b1a719a944ad1c Jul 08 01:43:55.462498 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:55.462385 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8aef35_e3f7_40a0_b597_01b98a5deba0.slice/crio-8cf37601f57c540c2aaf18bec90f4f58a3366929227c6386a97a53c2c17855e7 WatchSource:0}: Error finding container 8cf37601f57c540c2aaf18bec90f4f58a3366929227c6386a97a53c2c17855e7: Status 404 returned error can't find the container with id 8cf37601f57c540c2aaf18bec90f4f58a3366929227c6386a97a53c2c17855e7 Jul 08 01:43:55.463344 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:43:55.463280 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8266ac7a_4fa1_4a13_9ca3_49137e9303d0.slice/crio-6388f73abc3cfb40ebbacec05ef9d3598c86451d4c19e64a152a294d71150566 WatchSource:0}: Error finding container 6388f73abc3cfb40ebbacec05ef9d3598c86451d4c19e64a152a294d71150566: Status 404 returned error can't find the container with id 6388f73abc3cfb40ebbacec05ef9d3598c86451d4c19e64a152a294d71150566 Jul 08 01:43:55.477434 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.477393 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84kdl" event={"ID":"8266ac7a-4fa1-4a13-9ca3-49137e9303d0","Type":"ContainerStarted","Data":"6388f73abc3cfb40ebbacec05ef9d3598c86451d4c19e64a152a294d71150566"} Jul 08 01:43:55.480877 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.480843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerStarted","Data":"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9"} Jul 08 01:43:55.482053 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.482026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dscg2" event={"ID":"6e5523b0-6b1c-447e-b12c-3b1170259168","Type":"ContainerStarted","Data":"536922b33074b70f27216ae20a15d71f110ddea2beff6b98f6b1a719a944ad1c"} Jul 08 01:43:55.483175 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.483156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsjfh" event={"ID":"ec8aef35-e3f7-40a0-b597-01b98a5deba0","Type":"ContainerStarted","Data":"8cf37601f57c540c2aaf18bec90f4f58a3366929227c6386a97a53c2c17855e7"} Jul 08 01:43:55.530352 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:55.530309 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.31322823 podStartE2EDuration="6.530294667s" totalImagesPullingTime="4.200129211s" totalInitContainerRuntime="16.937226ms" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:55.528941212 +0000 UTC m=+66.983496856" watchObservedRunningTime="2026-07-08 01:43:55.530294667 +0000 UTC m=+66.984850312" Jul 08 01:43:56.490710 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:56.490659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" event={"ID":"8cddad98-596f-4423-b667-a980ad86c8f1","Type":"ContainerStarted","Data":"e1d2d70636af9b32b26d5cdb0c11169ea6d24bd50c9edc1ae1a4275ca2508e03"} Jul 08 01:43:56.508371 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:56.508308 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" podStartSLOduration=2.989628182 podStartE2EDuration="4.508283168s" totalImagesPullingTime="1.518654986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:43:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:56.506879571 +0000 UTC m=+67.961435231" watchObservedRunningTime="2026-07-08 01:43:56.508283168 +0000 UTC m=+67.962838815" Jul 08 01:43:57.501780 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:57.501654 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84kdl" event={"ID":"8266ac7a-4fa1-4a13-9ca3-49137e9303d0","Type":"ContainerStarted","Data":"b364229b7fe3654a202017c0ee7f8919a8f77704ce336cbd21a902fbcdeeb6ad"} Jul 08 01:43:57.501780 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:57.501746 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:57.501780 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:57.501772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84kdl" event={"ID":"8266ac7a-4fa1-4a13-9ca3-49137e9303d0","Type":"ContainerStarted","Data":"a018e34c07e9f88370eb3bdb7896dcc1a34c4d2006d049ac5cbf60989b75c981"} Jul 08 01:43:57.507178 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:57.507148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-n2rlt" Jul 08 01:43:57.521941 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:43:57.521887 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-84kdl" podStartSLOduration=66.886122453 podStartE2EDuration="1m8.521870201s" totalImagesPullingTime="1.635747748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:43:57.52052206 +0000 UTC m=+68.975077705" watchObservedRunningTime="2026-07-08 01:43:57.521870201 +0000 UTC m=+68.976425845" Jul 08 01:44:00.516336 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:00.516290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dscg2" event={"ID":"6e5523b0-6b1c-447e-b12c-3b1170259168","Type":"ContainerStarted","Data":"74c5f5197c5a6b7d2875db93b557d025846e1a416d14f8e104630192a3e442a1"} Jul 08 01:44:00.517872 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:00.517828 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsjfh" event={"ID":"ec8aef35-e3f7-40a0-b597-01b98a5deba0","Type":"ContainerStarted","Data":"c812b9eace1cfe720e22c4f1d418c156f3880ab6308e24ecaaf11d2c733625c1"} Jul 08 01:44:00.517984 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:00.517932 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:44:00.539876 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:00.539813 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dscg2" podStartSLOduration=66.697327769 podStartE2EDuration="1m11.539797681s" totalImagesPullingTime="4.842469912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:44:00.538886517 +0000 UTC m=+71.993442162" watchObservedRunningTime="2026-07-08 01:44:00.539797681 +0000 UTC m=+71.994353326" Jul 08 01:44:00.555430 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:00.555363 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tsjfh" podStartSLOduration=66.718556447 podStartE2EDuration="1m11.555342773s" totalImagesPullingTime="4.836786326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:44:00.553421331 +0000 UTC m=+72.007976975" watchObservedRunningTime="2026-07-08 01:44:00.555342773 +0000 UTC m=+72.009898420" Jul 08 01:44:31.524493 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:44:31.524381 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tsjfh" Jul 08 01:45:08.412662 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.412625 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.412982 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="alertmanager" containerID="cri-o://1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" gracePeriod=120 Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.413055 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-metric" containerID="cri-o://50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" gracePeriod=120 Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.413088 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="prom-label-proxy" containerID="cri-o://5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" gracePeriod=120 Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.413091 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy" containerID="cri-o://eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" gracePeriod=120 Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.413064 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-web" containerID="cri-o://f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" gracePeriod=120 Jul 08 01:45:08.413265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.413098 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="config-reloader" containerID="cri-o://fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" gracePeriod=120 Jul 08 01:45:08.761517 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761435 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" exitCode=0 Jul 08 01:45:08.761517 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761459 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" exitCode=0 Jul 08 01:45:08.761517 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761467 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" exitCode=0 Jul 08 01:45:08.761517 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761474 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" exitCode=0 Jul 08 01:45:08.761517 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761493 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9"} Jul 08 01:45:08.761770 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761547 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234"} Jul 08 01:45:08.761770 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761572 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece"} Jul 08 01:45:08.761770 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:08.761596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e"} Jul 08 01:45:09.659430 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.659403 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.744079 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.743996 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744079 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744043 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7t75\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744079 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744067 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744096 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744117 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744149 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744180 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744201 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744229 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744259 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744284 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744321 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.744349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.744346 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca\") pod \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\" (UID: \"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e\") " Jul 08 01:45:09.745265 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.745237 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:45:09.745639 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.745618 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 01:45:09.745862 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.745750 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 01:45:09.748672 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.748536 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume" (OuterVolumeSpecName: "config-volume") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.748672 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.748545 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out" (OuterVolumeSpecName: "config-out") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:45:09.749092 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.749046 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.749382 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.749353 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.749489 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.749381 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75" (OuterVolumeSpecName: "kube-api-access-m7t75") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "kube-api-access-m7t75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:45:09.749827 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.749783 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.749998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.749973 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.750210 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.750189 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:45:09.753033 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.753003 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.758379 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.758353 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config" (OuterVolumeSpecName: "web-config") pod "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" (UID: "5c09b0b7-1a7f-45f8-9864-f77a00e3db2e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 01:45:09.768568 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768540 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" exitCode=0 Jul 08 01:45:09.768568 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768566 2560 generic.go:356] "Generic (PLEG): container finished" podID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerID="f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" exitCode=0 Jul 08 01:45:09.768723 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf"} Jul 08 01:45:09.768723 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768633 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.768723 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e"} Jul 08 01:45:09.768723 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c09b0b7-1a7f-45f8-9864-f77a00e3db2e","Type":"ContainerDied","Data":"72c37dfdc375871a944e71341a870e854da0ebb8dc2d717e71b8b92e461a59ea"} Jul 08 01:45:09.768867 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.768738 2560 scope.go:117] "RemoveContainer" containerID="5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" Jul 08 01:45:09.776896 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.776871 2560 scope.go:117] "RemoveContainer" containerID="50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" Jul 08 01:45:09.783435 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.783418 2560 scope.go:117] "RemoveContainer" containerID="eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" Jul 08 01:45:09.790101 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.790079 2560 scope.go:117] "RemoveContainer" containerID="f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" Jul 08 01:45:09.791581 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.791562 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:09.795494 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.795475 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:09.797135 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.797122 2560 scope.go:117] "RemoveContainer" containerID="fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" Jul 08 01:45:09.803692 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.803677 2560 scope.go:117] "RemoveContainer" containerID="1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" Jul 08 01:45:09.810579 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.810546 2560 scope.go:117] "RemoveContainer" containerID="cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7" Jul 08 01:45:09.817411 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.817391 2560 scope.go:117] "RemoveContainer" containerID="5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" Jul 08 01:45:09.817667 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.817647 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9\": container with ID starting with 5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9 not found: ID does not exist" containerID="5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" Jul 08 01:45:09.817753 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.817677 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9"} err="failed to get container status \"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9\": rpc error: code = NotFound desc = could not find container \"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9\": container with ID starting with 5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9 not found: ID does not exist" Jul 08 01:45:09.817753 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.817711 2560 scope.go:117] "RemoveContainer" containerID="50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" Jul 08 01:45:09.817976 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.817952 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf\": container with ID starting with 50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf not found: ID does not exist" containerID="50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" Jul 08 01:45:09.818056 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.817988 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf"} err="failed to get container status \"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf\": rpc error: code = NotFound desc = could not find container \"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf\": container with ID starting with 50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf not found: ID does not exist" Jul 08 01:45:09.818056 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818012 2560 scope.go:117] "RemoveContainer" containerID="eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" Jul 08 01:45:09.818278 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.818261 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234\": container with ID starting with eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234 not found: ID does not exist" containerID="eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" Jul 08 01:45:09.818322 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818283 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234"} err="failed to get container status \"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234\": rpc error: code = NotFound desc = could not find container \"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234\": container with ID starting with eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234 not found: ID does not exist" Jul 08 01:45:09.818322 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818298 2560 scope.go:117] "RemoveContainer" containerID="f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" Jul 08 01:45:09.818526 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.818511 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e\": container with ID starting with f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e not found: ID does not exist" containerID="f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" Jul 08 01:45:09.818566 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818529 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e"} err="failed to get container status \"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e\": rpc error: code = NotFound desc = could not find container \"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e\": container with ID starting with f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e not found: ID does not exist" Jul 08 01:45:09.818566 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818540 2560 scope.go:117] "RemoveContainer" containerID="fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" Jul 08 01:45:09.818773 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.818751 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece\": container with ID starting with fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece not found: ID does not exist" containerID="fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" Jul 08 01:45:09.818856 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818780 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece"} err="failed to get container status \"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece\": rpc error: code = NotFound desc = could not find container \"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece\": container with ID starting with fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece not found: ID does not exist" Jul 08 01:45:09.818856 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.818800 2560 scope.go:117] "RemoveContainer" containerID="1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" Jul 08 01:45:09.819062 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.819046 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e\": container with ID starting with 1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e not found: ID does not exist" containerID="1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" Jul 08 01:45:09.819102 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819067 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e"} err="failed to get container status \"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e\": rpc error: code = NotFound desc = could not find container \"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e\": container with ID starting with 1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e not found: ID does not exist" Jul 08 01:45:09.819102 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819080 2560 scope.go:117] "RemoveContainer" containerID="cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7" Jul 08 01:45:09.819284 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:45:09.819268 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7\": container with ID starting with cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7 not found: ID does not exist" containerID="cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7" Jul 08 01:45:09.819320 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819288 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7"} err="failed to get container status \"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7\": rpc error: code = NotFound desc = could not find container \"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7\": container with ID starting with cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7 not found: ID does not exist" Jul 08 01:45:09.819320 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819304 2560 scope.go:117] "RemoveContainer" containerID="5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9" Jul 08 01:45:09.819527 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819508 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9"} err="failed to get container status \"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9\": rpc error: code = NotFound desc = could not find container \"5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9\": container with ID starting with 5bfaf45e6dcf5ba7497e2748cd1230c95524950fbcdbf875e11331c2cb7f62b9 not found: ID does not exist" Jul 08 01:45:09.819603 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819528 2560 scope.go:117] "RemoveContainer" containerID="50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf" Jul 08 01:45:09.819729 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819714 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf"} err="failed to get container status \"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf\": rpc error: code = NotFound desc = could not find container \"50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf\": container with ID starting with 50990d60568ca0a873054c51721e523cec15a9b8a518ba6b99ca682b68e9e8bf not found: ID does not exist" Jul 08 01:45:09.819771 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819729 2560 scope.go:117] "RemoveContainer" containerID="eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234" Jul 08 01:45:09.819911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819896 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234"} err="failed to get container status \"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234\": rpc error: code = NotFound desc = could not find container \"eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234\": container with ID starting with eccfc02e8d30bd3589a4a3c48d4f44eef804c67f5a08c70d10e4a0b9185e5234 not found: ID does not exist" Jul 08 01:45:09.819958 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.819911 2560 scope.go:117] "RemoveContainer" containerID="f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820242 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e"} err="failed to get container status \"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e\": rpc error: code = NotFound desc = could not find container \"f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e\": container with ID starting with f3f6593d806867063b5f35b90f7bc29c0d06432c648ad5b62f4be78f3d12573e not found: ID does not exist" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820264 2560 scope.go:117] "RemoveContainer" containerID="fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820521 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece"} err="failed to get container status \"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece\": rpc error: code = NotFound desc = could not find container \"fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece\": container with ID starting with fe7a82bad778857bf33c00c864387e4390ce9f0a9fc7756452e9698546a5fece not found: ID does not exist" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820539 2560 scope.go:117] "RemoveContainer" containerID="1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820774 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e"} err="failed to get container status \"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e\": rpc error: code = NotFound desc = could not find container \"1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e\": container with ID starting with 1ca3abb0911e54f37b614c8f6edbe76f0868ddb947a38c11a85c8dbfb8c61e4e not found: ID does not exist" Jul 08 01:45:09.822171 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.820790 2560 scope.go:117] "RemoveContainer" containerID="cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7" Jul 08 01:45:09.822438 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.822247 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7"} err="failed to get container status \"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7\": rpc error: code = NotFound desc = could not find container \"cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7\": container with ID starting with cf9927db4602506a17d735a0fb0585ea165babefdb392db739d7d6e4e85d59f7 not found: ID does not exist" Jul 08 01:45:09.824515 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824494 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:09.824748 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824734 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="alertmanager" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824750 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="alertmanager" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824758 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="config-reloader" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824764 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="config-reloader" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824775 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-web" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824781 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-web" Jul 08 01:45:09.824787 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824787 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-metric" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824793 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-metric" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824798 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="prom-label-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824804 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="prom-label-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824818 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="init-config-reloader" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824824 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="init-config-reloader" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824852 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824859 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824900 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-web" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824907 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824912 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="prom-label-proxy" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824921 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="config-reloader" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824929 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="kube-rbac-proxy-metric" Jul 08 01:45:09.824994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.824934 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" containerName="alertmanager" Jul 08 01:45:09.828194 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.828177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.830601 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830575 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 01:45:09.830707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830614 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 01:45:09.830707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 01:45:09.830707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830618 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 01:45:09.830707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830640 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 01:45:09.830980 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830959 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 01:45:09.830980 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830971 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 01:45:09.830980 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830974 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 01:45:09.831157 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.830959 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nz77x\"" Jul 08 01:45:09.836112 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.836091 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 01:45:09.843376 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.843355 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:09.845471 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ch9q\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-kube-api-access-7ch9q\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845544 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845581 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845616 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845616 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845688 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845688 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845739 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845759 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845890 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845890 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845890 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845893 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-cluster-tls-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845902 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-tls-assets\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845910 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-web-config\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845930 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-main-db\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845950 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845964 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.845988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845977 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.845990 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-out\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.846002 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-metrics-client-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.846014 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-config-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.846031 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7t75\" (UniqueName: \"kubernetes.io/projected/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-kube-api-access-m7t75\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.846045 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.846213 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.846062 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e-secret-alertmanager-main-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:45:09.946693 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946693 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946985 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946985 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946985 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ch9q\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-kube-api-access-7ch9q\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946985 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.946985 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.946942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947239 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947162 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947288 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947344 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947412 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947412 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947370 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947412 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947405 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947562 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.947612 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.947590 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.948297 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.948271 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.949631 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.949602 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-out\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950010 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950124 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950232 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950287 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950506 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950288 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.950560 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.950544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.951717 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.951702 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.951775 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.951740 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-web-config\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:09.954981 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:09.954961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ch9q\" (UniqueName: \"kubernetes.io/projected/0c759fae-ab4d-40d8-ba3f-096e30bc7a4a-kube-api-access-7ch9q\") pod \"alertmanager-main-0\" (UID: \"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:10.140273 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:10.140240 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 01:45:10.276767 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:10.276733 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 01:45:10.279731 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:45:10.279701 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c759fae_ab4d_40d8_ba3f_096e30bc7a4a.slice/crio-094ecdca53d6613af30f2ffd68fa5bc3331cc9d13aaadf8577e8674218af3f5a WatchSource:0}: Error finding container 094ecdca53d6613af30f2ffd68fa5bc3331cc9d13aaadf8577e8674218af3f5a: Status 404 returned error can't find the container with id 094ecdca53d6613af30f2ffd68fa5bc3331cc9d13aaadf8577e8674218af3f5a Jul 08 01:45:10.773977 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:10.773895 2560 generic.go:356] "Generic (PLEG): container finished" podID="0c759fae-ab4d-40d8-ba3f-096e30bc7a4a" containerID="38dfdee1ffd689f7dc3eaccde3f60e44bd87c27d728c8c209c4bee9310c07b2b" exitCode=0 Jul 08 01:45:10.773977 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:10.773933 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerDied","Data":"38dfdee1ffd689f7dc3eaccde3f60e44bd87c27d728c8c209c4bee9310c07b2b"} Jul 08 01:45:10.773977 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:10.773980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"094ecdca53d6613af30f2ffd68fa5bc3331cc9d13aaadf8577e8674218af3f5a"} Jul 08 01:45:11.033504 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.033478 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c09b0b7-1a7f-45f8-9864-f77a00e3db2e" path="/var/lib/kubelet/pods/5c09b0b7-1a7f-45f8-9864-f77a00e3db2e/volumes" Jul 08 01:45:11.780656 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780621 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"93197bd1d3b70cdd7e96c3e09e1df4af7351501245e726c32797ac120bb6a37a"} Jul 08 01:45:11.781062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"2e0e9b70e7cb49acda4cffea2ba5e405065a3c663c93ad46c3af2ccaed6ed8c4"} Jul 08 01:45:11.781062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"7ffcea15a0b78fdb2a990b0c925e2727eab636eccb3c194f97a73c23f46ea886"} Jul 08 01:45:11.781062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"08ea8f83c5d067f62a1e92bec81e6bd3d67d3c1bfbcb1f139e8ee77600a78912"} Jul 08 01:45:11.781062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"35ae8b2256dfb364061825f878c1c3ef398f1696c0f0d486a35bc46f7a636851"} Jul 08 01:45:11.781062 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.780785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c759fae-ab4d-40d8-ba3f-096e30bc7a4a","Type":"ContainerStarted","Data":"8350ea9702c62944271018efab307162d8850c612f2a03c642f770ef90024767"} Jul 08 01:45:11.805955 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:11.805898 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.790131327 podStartE2EDuration="2.805882802s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.751475ms" isStatefulPod=false podCreationTimestamp="2026-07-08 01:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:45:11.804505955 +0000 UTC m=+143.259061611" watchObservedRunningTime="2026-07-08 01:45:11.805882802 +0000 UTC m=+143.260438447" Jul 08 01:45:12.440034 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.439992 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c89c99c66-wx6k4"] Jul 08 01:45:12.442750 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.442725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.444988 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.444963 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 01:45:12.445501 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.445479 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 01:45:12.445707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.445692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 01:45:12.445928 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.445913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 01:45:12.446105 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.446092 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-62vt7\"" Jul 08 01:45:12.446300 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.446281 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 01:45:12.450357 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.450326 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 01:45:12.456180 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.456152 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c89c99c66-wx6k4"] Jul 08 01:45:12.466101 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-serving-certs-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466243 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-metrics-client-ca\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466243 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj6qb\" (UniqueName: \"kubernetes.io/projected/5f231b5d-22a4-41d2-99ec-452315f1f165-kube-api-access-mj6qb\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466243 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-federate-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466243 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466231 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466460 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466247 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466460 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466267 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.466460 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.466298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.567552 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-federate-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.567763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.567763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.567763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.567763 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-serving-certs-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-metrics-client-ca\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.567915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj6qb\" (UniqueName: \"kubernetes.io/projected/5f231b5d-22a4-41d2-99ec-452315f1f165-kube-api-access-mj6qb\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568653 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.568613 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-serving-certs-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568855 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.568805 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.568925 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.568874 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f231b5d-22a4-41d2-99ec-452315f1f165-metrics-client-ca\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.571940 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.571291 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-federate-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.571940 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.571389 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-telemeter-client-tls\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.571940 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.571674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.572603 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.572582 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f231b5d-22a4-41d2-99ec-452315f1f165-secret-telemeter-client\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.576065 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.576049 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj6qb\" (UniqueName: \"kubernetes.io/projected/5f231b5d-22a4-41d2-99ec-452315f1f165-kube-api-access-mj6qb\") pod \"telemeter-client-c89c99c66-wx6k4\" (UID: \"5f231b5d-22a4-41d2-99ec-452315f1f165\") " pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.754330 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.754225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" Jul 08 01:45:12.907119 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:12.907088 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c89c99c66-wx6k4"] Jul 08 01:45:12.909331 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:45:12.909304 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f231b5d_22a4_41d2_99ec_452315f1f165.slice/crio-d5e3bc655198794b4cb9158547143f5c6111150440bfff8b398e224fd8ec3c59 WatchSource:0}: Error finding container d5e3bc655198794b4cb9158547143f5c6111150440bfff8b398e224fd8ec3c59: Status 404 returned error can't find the container with id d5e3bc655198794b4cb9158547143f5c6111150440bfff8b398e224fd8ec3c59 Jul 08 01:45:13.793974 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:13.793936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" event={"ID":"5f231b5d-22a4-41d2-99ec-452315f1f165","Type":"ContainerStarted","Data":"d5e3bc655198794b4cb9158547143f5c6111150440bfff8b398e224fd8ec3c59"} Jul 08 01:45:15.804295 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:15.804256 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" event={"ID":"5f231b5d-22a4-41d2-99ec-452315f1f165","Type":"ContainerStarted","Data":"f5e0d2d478f6774df6b0e87a333bbeb5f9ae18aec4cc9454f7bc6beb2b5aa299"} Jul 08 01:45:15.804765 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:15.804321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" event={"ID":"5f231b5d-22a4-41d2-99ec-452315f1f165","Type":"ContainerStarted","Data":"df61f918a3150a3aadf2dd21eda8a093beb36df5178bddd4bef1b675055138f9"} Jul 08 01:45:15.804765 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:15.804346 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" event={"ID":"5f231b5d-22a4-41d2-99ec-452315f1f165","Type":"ContainerStarted","Data":"2bbdb4f3311c47dc4c3600d9dbd71dbd1a9924eafd2dfe2a0225049a0ea79099"} Jul 08 01:45:15.831283 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:45:15.831223 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c89c99c66-wx6k4" podStartSLOduration=1.966677288 podStartE2EDuration="3.831204857s" totalImagesPullingTime="1.864527569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:45:15.829188415 +0000 UTC m=+147.283744065" watchObservedRunningTime="2026-07-08 01:45:15.831204857 +0000 UTC m=+147.285760501" Jul 08 01:47:44.813035 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.813000 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc"] Jul 08 01:47:44.816087 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.816062 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:44.818618 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.818597 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 01:47:44.818734 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.818676 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 01:47:44.818792 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.818749 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlrz5\"" Jul 08 01:47:44.826514 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.826491 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc"] Jul 08 01:47:44.927349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.927307 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8h5x\" (UniqueName: \"kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:44.927349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.927344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:44.927612 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:44.927381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.028072 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.028044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.028228 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.028101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8h5x\" (UniqueName: \"kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.028228 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.028121 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.028396 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.028380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.028446 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.028408 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.037366 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.037334 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8h5x\" (UniqueName: \"kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.124726 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.124642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:47:45.251786 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.251758 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc"] Jul 08 01:47:45.254490 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:47:45.254463 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0560ddb3_f802_4d2d_8bb7_335847ef03c7.slice/crio-864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c WatchSource:0}: Error finding container 864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c: Status 404 returned error can't find the container with id 864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c Jul 08 01:47:45.300099 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:45.300064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerStarted","Data":"864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c"} Jul 08 01:47:48.891205 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:48.891177 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 01:47:52.328563 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:52.328531 2560 generic.go:356] "Generic (PLEG): container finished" podID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerID="50d66ef8266fb5c718e937d48cec5a38dfa79b8434d122e859f676ad5d3eafd3" exitCode=0 Jul 08 01:47:52.329048 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:52.328624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerDied","Data":"50d66ef8266fb5c718e937d48cec5a38dfa79b8434d122e859f676ad5d3eafd3"} Jul 08 01:47:52.329573 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:52.329558 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 01:47:54.337458 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:54.337428 2560 generic.go:356] "Generic (PLEG): container finished" podID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerID="ce0ae651e2c698cd09ee7dd9803f37393c8b2fdb7c0e17d2fe59cfdb129bb5cf" exitCode=0 Jul 08 01:47:54.337766 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:47:54.337481 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerDied","Data":"ce0ae651e2c698cd09ee7dd9803f37393c8b2fdb7c0e17d2fe59cfdb129bb5cf"} Jul 08 01:48:02.371431 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:02.371386 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerStarted","Data":"e3506c4c83543294550d98fe968ff4f219ca74fd6f76f2b9f8749b068a7d310d"} Jul 08 01:48:02.396138 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:02.396082 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" podStartSLOduration=2.237663644 podStartE2EDuration="18.396063519s" totalImagesPullingTime="16.158399875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:47:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:02.394764701 +0000 UTC m=+313.849320346" watchObservedRunningTime="2026-07-08 01:48:02.396063519 +0000 UTC m=+313.850619164" Jul 08 01:48:03.376432 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:03.376384 2560 generic.go:356] "Generic (PLEG): container finished" podID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerID="e3506c4c83543294550d98fe968ff4f219ca74fd6f76f2b9f8749b068a7d310d" exitCode=0 Jul 08 01:48:03.376803 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:03.376460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerDied","Data":"e3506c4c83543294550d98fe968ff4f219ca74fd6f76f2b9f8749b068a7d310d"} Jul 08 01:48:04.505146 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.505119 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:48:04.593952 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.593905 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle\") pod \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " Jul 08 01:48:04.594141 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.593987 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util\") pod \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " Jul 08 01:48:04.594141 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.594021 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8h5x\" (UniqueName: \"kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x\") pod \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\" (UID: \"0560ddb3-f802-4d2d-8bb7-335847ef03c7\") " Jul 08 01:48:04.594655 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.594630 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle" (OuterVolumeSpecName: "bundle") pod "0560ddb3-f802-4d2d-8bb7-335847ef03c7" (UID: "0560ddb3-f802-4d2d-8bb7-335847ef03c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:04.596219 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.596198 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x" (OuterVolumeSpecName: "kube-api-access-g8h5x") pod "0560ddb3-f802-4d2d-8bb7-335847ef03c7" (UID: "0560ddb3-f802-4d2d-8bb7-335847ef03c7"). InnerVolumeSpecName "kube-api-access-g8h5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:48:04.599622 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.599595 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util" (OuterVolumeSpecName: "util") pod "0560ddb3-f802-4d2d-8bb7-335847ef03c7" (UID: "0560ddb3-f802-4d2d-8bb7-335847ef03c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:04.695407 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.695314 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:04.695407 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.695346 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0560ddb3-f802-4d2d-8bb7-335847ef03c7-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:04.695407 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:04.695355 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8h5x\" (UniqueName: \"kubernetes.io/projected/0560ddb3-f802-4d2d-8bb7-335847ef03c7-kube-api-access-g8h5x\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:05.386798 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:05.386769 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" Jul 08 01:48:05.386990 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:05.386775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vvqgc" event={"ID":"0560ddb3-f802-4d2d-8bb7-335847ef03c7","Type":"ContainerDied","Data":"864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c"} Jul 08 01:48:05.386990 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:05.386888 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="864b33874b038513300b24241e3b183ec7177f421ff54ea1df1e5e41eb5e742c" Jul 08 01:48:13.008486 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008449 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2"] Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008821 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="util" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008857 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="util" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008880 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="pull" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008889 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="pull" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008897 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="extract" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008906 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="extract" Jul 08 01:48:13.009114 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.008968 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0560ddb3-f802-4d2d-8bb7-335847ef03c7" containerName="extract" Jul 08 01:48:13.011847 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.011813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.014408 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.014384 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-psmk4\"" Jul 08 01:48:13.014645 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.014628 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 01:48:13.014721 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.014667 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 01:48:13.029058 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.029026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2"] Jul 08 01:48:13.160014 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.159961 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c0556df-c050-44c5-8f30-9ccbae79c2d8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.160222 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.160131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w8qn\" (UniqueName: \"kubernetes.io/projected/8c0556df-c050-44c5-8f30-9ccbae79c2d8-kube-api-access-5w8qn\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.260677 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.260586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w8qn\" (UniqueName: \"kubernetes.io/projected/8c0556df-c050-44c5-8f30-9ccbae79c2d8-kube-api-access-5w8qn\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.260677 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.260625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c0556df-c050-44c5-8f30-9ccbae79c2d8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.261013 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.260996 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c0556df-c050-44c5-8f30-9ccbae79c2d8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.269233 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.269212 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w8qn\" (UniqueName: \"kubernetes.io/projected/8c0556df-c050-44c5-8f30-9ccbae79c2d8-kube-api-access-5w8qn\") pod \"cert-manager-operator-controller-manager-55c444bf86-fh5x2\" (UID: \"8c0556df-c050-44c5-8f30-9ccbae79c2d8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.321242 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.321207 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" Jul 08 01:48:13.458646 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:13.458612 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2"] Jul 08 01:48:13.462775 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:13.462735 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0556df_c050_44c5_8f30_9ccbae79c2d8.slice/crio-038d8e06619a6f016bd1b99c4573b7535753205b7d2d932e5d3f521a87d4bdbf WatchSource:0}: Error finding container 038d8e06619a6f016bd1b99c4573b7535753205b7d2d932e5d3f521a87d4bdbf: Status 404 returned error can't find the container with id 038d8e06619a6f016bd1b99c4573b7535753205b7d2d932e5d3f521a87d4bdbf Jul 08 01:48:14.422643 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:14.422598 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" event={"ID":"8c0556df-c050-44c5-8f30-9ccbae79c2d8","Type":"ContainerStarted","Data":"038d8e06619a6f016bd1b99c4573b7535753205b7d2d932e5d3f521a87d4bdbf"} Jul 08 01:48:15.428014 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:15.427920 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" event={"ID":"8c0556df-c050-44c5-8f30-9ccbae79c2d8","Type":"ContainerStarted","Data":"8550291f58bea9e2f34c127174e131812b1a5487ba1b1e3e8f3c09c61ddefc8b"} Jul 08 01:48:15.451596 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:15.451543 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fh5x2" podStartSLOduration=1.7502886420000001 podStartE2EDuration="3.451524891s" totalImagesPullingTime="1.701236249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:15.449104909 +0000 UTC m=+326.903660554" watchObservedRunningTime="2026-07-08 01:48:15.451524891 +0000 UTC m=+326.906080535" Jul 08 01:48:16.740130 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.740095 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr"] Jul 08 01:48:16.743579 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.743559 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.747147 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.747126 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 01:48:16.747266 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.747126 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlrz5\"" Jul 08 01:48:16.750686 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.750670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 01:48:16.763727 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.763700 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr"] Jul 08 01:48:16.889016 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.888978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.889216 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.889043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.889216 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.889084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtmk7\" (UniqueName: \"kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.990120 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.990086 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.990349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.990130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.990349 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.990158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtmk7\" (UniqueName: \"kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.990500 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.990477 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:16.990566 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:16.990537 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:17.009534 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.009497 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtmk7\" (UniqueName: \"kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:17.052880 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.052807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:17.071163 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.071125 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hwj7d"] Jul 08 01:48:17.074806 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.074786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.080392 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.080363 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 01:48:17.080524 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.080408 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-k4mps\"" Jul 08 01:48:17.080816 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.080795 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 01:48:17.092424 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.092398 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hwj7d"] Jul 08 01:48:17.189684 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.189647 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr"] Jul 08 01:48:17.191909 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.191879 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.192020 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.191913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l55qs\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-kube-api-access-l55qs\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.192541 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:17.192521 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34c8f70_93f0_442d_b476_3263e49cd383.slice/crio-a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7 WatchSource:0}: Error finding container a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7: Status 404 returned error can't find the container with id a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7 Jul 08 01:48:17.292643 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.292611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.292767 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.292658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l55qs\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-kube-api-access-l55qs\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.301523 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.301493 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.301633 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.301592 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l55qs\" (UniqueName: \"kubernetes.io/projected/e562d9bb-dadc-410c-a9a1-ae99a73e3c29-kube-api-access-l55qs\") pod \"cert-manager-webhook-6486ffd997-hwj7d\" (UID: \"e562d9bb-dadc-410c-a9a1-ae99a73e3c29\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.394984 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.394941 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:17.439450 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.439411 2560 generic.go:356] "Generic (PLEG): container finished" podID="d34c8f70-93f0-442d-b476-3263e49cd383" containerID="bdd99f650fa4e3978617d754ca30750dda79987ccc77bd85ff25afea518a2448" exitCode=0 Jul 08 01:48:17.439621 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.439485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" event={"ID":"d34c8f70-93f0-442d-b476-3263e49cd383","Type":"ContainerDied","Data":"bdd99f650fa4e3978617d754ca30750dda79987ccc77bd85ff25afea518a2448"} Jul 08 01:48:17.439621 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.439541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" event={"ID":"d34c8f70-93f0-442d-b476-3263e49cd383","Type":"ContainerStarted","Data":"a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7"} Jul 08 01:48:17.533487 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:17.533411 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hwj7d"] Jul 08 01:48:17.536333 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:17.536301 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode562d9bb_dadc_410c_a9a1_ae99a73e3c29.slice/crio-927e2b0d5602ef71c66c87bfd241a10db7905e73fb5b6134a740842d9abd5e58 WatchSource:0}: Error finding container 927e2b0d5602ef71c66c87bfd241a10db7905e73fb5b6134a740842d9abd5e58: Status 404 returned error can't find the container with id 927e2b0d5602ef71c66c87bfd241a10db7905e73fb5b6134a740842d9abd5e58 Jul 08 01:48:18.446999 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:18.446937 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" event={"ID":"e562d9bb-dadc-410c-a9a1-ae99a73e3c29","Type":"ContainerStarted","Data":"927e2b0d5602ef71c66c87bfd241a10db7905e73fb5b6134a740842d9abd5e58"} Jul 08 01:48:19.636634 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.636589 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t"] Jul 08 01:48:19.640411 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.640388 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.643404 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.643382 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xp527\"" Jul 08 01:48:19.653822 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.653790 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t"] Jul 08 01:48:19.816814 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.816771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.816997 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.816854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmlzq\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-kube-api-access-lmlzq\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.918395 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.918284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.918395 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.918366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmlzq\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-kube-api-access-lmlzq\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.926740 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.926703 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.926912 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.926745 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmlzq\" (UniqueName: \"kubernetes.io/projected/4ace18c2-f057-4535-9539-f38bfdbd8347-kube-api-access-lmlzq\") pod \"cert-manager-cainjector-559cdd8c87-d8r2t\" (UID: \"4ace18c2-f057-4535-9539-f38bfdbd8347\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:19.949432 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:19.949394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" Jul 08 01:48:20.770204 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:20.770171 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t"] Jul 08 01:48:20.773063 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:20.773036 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ace18c2_f057_4535_9539_f38bfdbd8347.slice/crio-61fdff8c511f055a3389320f66aca37b5cb73336f88e2130f2334e6d8cc95554 WatchSource:0}: Error finding container 61fdff8c511f055a3389320f66aca37b5cb73336f88e2130f2334e6d8cc95554: Status 404 returned error can't find the container with id 61fdff8c511f055a3389320f66aca37b5cb73336f88e2130f2334e6d8cc95554 Jul 08 01:48:21.462008 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.461974 2560 generic.go:356] "Generic (PLEG): container finished" podID="d34c8f70-93f0-442d-b476-3263e49cd383" containerID="76f6a8253c7510e72062af409bbea85bf809459d58b60688faa6fe6b8154beb7" exitCode=0 Jul 08 01:48:21.462211 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.462064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" event={"ID":"d34c8f70-93f0-442d-b476-3263e49cd383","Type":"ContainerDied","Data":"76f6a8253c7510e72062af409bbea85bf809459d58b60688faa6fe6b8154beb7"} Jul 08 01:48:21.463504 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.463481 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" event={"ID":"e562d9bb-dadc-410c-a9a1-ae99a73e3c29","Type":"ContainerStarted","Data":"1ba4c782d964c4c1552dcd12ce2120d40eef3eb16eb15c9cbc7d4e070b703093"} Jul 08 01:48:21.463612 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.463532 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:21.464994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.464970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" event={"ID":"4ace18c2-f057-4535-9539-f38bfdbd8347","Type":"ContainerStarted","Data":"fd8761b68fbc51149dda85bd35afed934b94dab8e133ce3f73b1027fa177d15b"} Jul 08 01:48:21.465089 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.465001 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" event={"ID":"4ace18c2-f057-4535-9539-f38bfdbd8347","Type":"ContainerStarted","Data":"61fdff8c511f055a3389320f66aca37b5cb73336f88e2130f2334e6d8cc95554"} Jul 08 01:48:21.496069 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.496020 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" podStartSLOduration=1.399922069 podStartE2EDuration="4.496003632s" totalImagesPullingTime="3.096081563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:21.494856409 +0000 UTC m=+332.949412057" watchObservedRunningTime="2026-07-08 01:48:21.496003632 +0000 UTC m=+332.950559276" Jul 08 01:48:21.509779 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:21.509734 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d8r2t" podStartSLOduration=2.509719396 podStartE2EDuration="2.509719396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:21.508962701 +0000 UTC m=+332.963518347" watchObservedRunningTime="2026-07-08 01:48:21.509719396 +0000 UTC m=+332.964275036" Jul 08 01:48:22.471329 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:22.471291 2560 generic.go:356] "Generic (PLEG): container finished" podID="d34c8f70-93f0-442d-b476-3263e49cd383" containerID="9ea4e583efa66ed7f8a5fb5d0fd525b69ed16d002792d9fce85acffa0738cbd2" exitCode=0 Jul 08 01:48:22.471765 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:22.471372 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" event={"ID":"d34c8f70-93f0-442d-b476-3263e49cd383","Type":"ContainerDied","Data":"9ea4e583efa66ed7f8a5fb5d0fd525b69ed16d002792d9fce85acffa0738cbd2"} Jul 08 01:48:23.610416 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.610382 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:23.749715 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.749643 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle\") pod \"d34c8f70-93f0-442d-b476-3263e49cd383\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " Jul 08 01:48:23.749715 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.749695 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util\") pod \"d34c8f70-93f0-442d-b476-3263e49cd383\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " Jul 08 01:48:23.749932 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.749732 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtmk7\" (UniqueName: \"kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7\") pod \"d34c8f70-93f0-442d-b476-3263e49cd383\" (UID: \"d34c8f70-93f0-442d-b476-3263e49cd383\") " Jul 08 01:48:23.750154 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.750124 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle" (OuterVolumeSpecName: "bundle") pod "d34c8f70-93f0-442d-b476-3263e49cd383" (UID: "d34c8f70-93f0-442d-b476-3263e49cd383"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:23.751896 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.751863 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7" (OuterVolumeSpecName: "kube-api-access-jtmk7") pod "d34c8f70-93f0-442d-b476-3263e49cd383" (UID: "d34c8f70-93f0-442d-b476-3263e49cd383"). InnerVolumeSpecName "kube-api-access-jtmk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:48:23.753769 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.753746 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util" (OuterVolumeSpecName: "util") pod "d34c8f70-93f0-442d-b476-3263e49cd383" (UID: "d34c8f70-93f0-442d-b476-3263e49cd383"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:23.851033 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.850994 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:23.851033 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.851023 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d34c8f70-93f0-442d-b476-3263e49cd383-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:23.851033 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:23.851034 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtmk7\" (UniqueName: \"kubernetes.io/projected/d34c8f70-93f0-442d-b476-3263e49cd383-kube-api-access-jtmk7\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:24.485995 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:24.485967 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" Jul 08 01:48:24.485995 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:24.485978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhlcrr" event={"ID":"d34c8f70-93f0-442d-b476-3263e49cd383","Type":"ContainerDied","Data":"a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7"} Jul 08 01:48:24.486198 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:24.486041 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a990658aaa20aecb1d4e2c300129eb206652d419302ef5614a5622d409ff1de7" Jul 08 01:48:27.473978 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:27.473946 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hwj7d" Jul 08 01:48:30.588868 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.588819 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd"] Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589144 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="util" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589156 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="util" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589173 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="pull" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589180 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="pull" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589195 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="extract" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589202 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="extract" Jul 08 01:48:30.589261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.589258 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d34c8f70-93f0-442d-b476-3263e49cd383" containerName="extract" Jul 08 01:48:30.591791 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.591773 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.594238 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.594216 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 01:48:30.595163 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.595144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5tbqm\"" Jul 08 01:48:30.595163 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.595153 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 01:48:30.600738 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.600718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd"] Jul 08 01:48:30.706941 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.706899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdd4t\" (UniqueName: \"kubernetes.io/projected/5aac7308-c51a-4d77-bc97-0f601f9f9c27-kube-api-access-zdd4t\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.706941 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.706946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5aac7308-c51a-4d77-bc97-0f601f9f9c27-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.808173 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.808134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdd4t\" (UniqueName: \"kubernetes.io/projected/5aac7308-c51a-4d77-bc97-0f601f9f9c27-kube-api-access-zdd4t\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.808173 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.808176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5aac7308-c51a-4d77-bc97-0f601f9f9c27-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.808578 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.808562 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5aac7308-c51a-4d77-bc97-0f601f9f9c27-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.817422 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.817397 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdd4t\" (UniqueName: \"kubernetes.io/projected/5aac7308-c51a-4d77-bc97-0f601f9f9c27-kube-api-access-zdd4t\") pod \"openshift-lws-operator-fd8ccff4c-5ccsd\" (UID: \"5aac7308-c51a-4d77-bc97-0f601f9f9c27\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:30.900707 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:30.900616 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" Jul 08 01:48:31.028709 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:31.028685 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd"] Jul 08 01:48:31.031195 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:31.031165 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5aac7308_c51a_4d77_bc97_0f601f9f9c27.slice/crio-35b62a37bd937b67102e794f2e9510e7fa8af9940ae30702592dec96d382b5db WatchSource:0}: Error finding container 35b62a37bd937b67102e794f2e9510e7fa8af9940ae30702592dec96d382b5db: Status 404 returned error can't find the container with id 35b62a37bd937b67102e794f2e9510e7fa8af9940ae30702592dec96d382b5db Jul 08 01:48:31.518099 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:31.518063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" event={"ID":"5aac7308-c51a-4d77-bc97-0f601f9f9c27","Type":"ContainerStarted","Data":"35b62a37bd937b67102e794f2e9510e7fa8af9940ae30702592dec96d382b5db"} Jul 08 01:48:34.532911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:34.532879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" event={"ID":"5aac7308-c51a-4d77-bc97-0f601f9f9c27","Type":"ContainerStarted","Data":"e0e144f71a4f170746d0a04bd4f6a017ae50a99dbbd783a2f12c325182f3fc1a"} Jul 08 01:48:34.549483 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:34.549432 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5ccsd" podStartSLOduration=1.25154455 podStartE2EDuration="4.549417095s" totalImagesPullingTime="3.297872545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:34.54821464 +0000 UTC m=+346.002770286" watchObservedRunningTime="2026-07-08 01:48:34.549417095 +0000 UTC m=+346.003972740" Jul 08 01:48:37.045869 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.045820 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg"] Jul 08 01:48:37.066107 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.066074 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg"] Jul 08 01:48:37.066251 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.066163 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.069590 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.069553 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 01:48:37.069590 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.069576 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zlrz5\"" Jul 08 01:48:37.070119 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.070101 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 01:48:37.158434 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.158396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.158434 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.158442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.158644 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.158465 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snb7k\" (UniqueName: \"kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.258966 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.258936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.259170 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.258978 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.259170 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.259011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snb7k\" (UniqueName: \"kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.259331 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.259312 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.259411 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.259388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.267693 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.267665 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snb7k\" (UniqueName: \"kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.374851 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.374804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:37.526415 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.526381 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg"] Jul 08 01:48:37.530365 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:37.530341 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f17ffa_d256_4983_820b_07035a93a7e5.slice/crio-b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6 WatchSource:0}: Error finding container b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6: Status 404 returned error can't find the container with id b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6 Jul 08 01:48:37.546250 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:37.546221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" event={"ID":"16f17ffa-d256-4983-820b-07035a93a7e5","Type":"ContainerStarted","Data":"b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6"} Jul 08 01:48:38.551683 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:38.551589 2560 generic.go:356] "Generic (PLEG): container finished" podID="16f17ffa-d256-4983-820b-07035a93a7e5" containerID="16900c8b43f4dbb51419880c28f43314f774669fe2a5ff1e112a883bbbd1bc83" exitCode=0 Jul 08 01:48:38.551683 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:38.551644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" event={"ID":"16f17ffa-d256-4983-820b-07035a93a7e5","Type":"ContainerDied","Data":"16900c8b43f4dbb51419880c28f43314f774669fe2a5ff1e112a883bbbd1bc83"} Jul 08 01:48:39.557309 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:39.557268 2560 generic.go:356] "Generic (PLEG): container finished" podID="16f17ffa-d256-4983-820b-07035a93a7e5" containerID="c45e94645cd1aa4358f63527b1a9d46c0f32a031a9c62f0d6deba201bac5dbbe" exitCode=0 Jul 08 01:48:39.557697 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:39.557325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" event={"ID":"16f17ffa-d256-4983-820b-07035a93a7e5","Type":"ContainerDied","Data":"c45e94645cd1aa4358f63527b1a9d46c0f32a031a9c62f0d6deba201bac5dbbe"} Jul 08 01:48:40.563628 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:40.563587 2560 generic.go:356] "Generic (PLEG): container finished" podID="16f17ffa-d256-4983-820b-07035a93a7e5" containerID="2aa7a6b794e57fec553811c0301e6b353ce1f861bb38723805e626b9a74e0c9e" exitCode=0 Jul 08 01:48:40.563998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:40.563638 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" event={"ID":"16f17ffa-d256-4983-820b-07035a93a7e5","Type":"ContainerDied","Data":"2aa7a6b794e57fec553811c0301e6b353ce1f861bb38723805e626b9a74e0c9e"} Jul 08 01:48:41.695501 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.695476 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:41.792897 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.792864 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle\") pod \"16f17ffa-d256-4983-820b-07035a93a7e5\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " Jul 08 01:48:41.793057 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.792955 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util\") pod \"16f17ffa-d256-4983-820b-07035a93a7e5\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " Jul 08 01:48:41.793057 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.793006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snb7k\" (UniqueName: \"kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k\") pod \"16f17ffa-d256-4983-820b-07035a93a7e5\" (UID: \"16f17ffa-d256-4983-820b-07035a93a7e5\") " Jul 08 01:48:41.793663 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.793638 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle" (OuterVolumeSpecName: "bundle") pod "16f17ffa-d256-4983-820b-07035a93a7e5" (UID: "16f17ffa-d256-4983-820b-07035a93a7e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:41.795162 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.795141 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k" (OuterVolumeSpecName: "kube-api-access-snb7k") pod "16f17ffa-d256-4983-820b-07035a93a7e5" (UID: "16f17ffa-d256-4983-820b-07035a93a7e5"). InnerVolumeSpecName "kube-api-access-snb7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:48:41.801261 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.801236 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util" (OuterVolumeSpecName: "util") pod "16f17ffa-d256-4983-820b-07035a93a7e5" (UID: "16f17ffa-d256-4983-820b-07035a93a7e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:48:41.894042 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.893957 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:41.894042 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.893987 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snb7k\" (UniqueName: \"kubernetes.io/projected/16f17ffa-d256-4983-820b-07035a93a7e5-kube-api-access-snb7k\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:41.894042 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:41.893997 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16f17ffa-d256-4983-820b-07035a93a7e5-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:48:42.574761 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:42.574736 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" Jul 08 01:48:42.574761 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:42.574742 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftvffg" event={"ID":"16f17ffa-d256-4983-820b-07035a93a7e5","Type":"ContainerDied","Data":"b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6"} Jul 08 01:48:42.574981 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:42.574783 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3dfefa4abd7282702b7e246042cc0ab6ff91bd529c12d2444905608bac8cfc6" Jul 08 01:48:50.292040 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292005 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4"] Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292400 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="util" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292420 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="util" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292448 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="pull" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292453 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="pull" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292460 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="extract" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292465 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="extract" Jul 08 01:48:50.292559 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.292528 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="16f17ffa-d256-4983-820b-07035a93a7e5" containerName="extract" Jul 08 01:48:50.295122 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.295098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.299021 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.298998 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 01:48:50.299250 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.299235 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 01:48:50.299576 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.299560 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2dr79\"" Jul 08 01:48:50.299576 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.299568 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 01:48:50.302298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.302283 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 01:48:50.315953 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.315929 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4"] Jul 08 01:48:50.355998 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.355957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.356157 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.356071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-manifests\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.356157 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.356098 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.356157 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.356119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxqw\" (UniqueName: \"kubernetes.io/projected/d984c7b0-936c-4728-8cbd-be31e836a65b-kube-api-access-7nxqw\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.356340 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.356160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-tmp\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.456803 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.456768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.457023 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.456851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-manifests\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.457023 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.456878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.457023 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.456896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxqw\" (UniqueName: \"kubernetes.io/projected/d984c7b0-936c-4728-8cbd-be31e836a65b-kube-api-access-7nxqw\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.457023 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.456927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-tmp\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.457325 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.457302 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-manifests\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.459142 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.459118 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d984c7b0-936c-4728-8cbd-be31e836a65b-tmp\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.459368 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.459348 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.459498 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.459478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d984c7b0-936c-4728-8cbd-be31e836a65b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.466011 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.465990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxqw\" (UniqueName: \"kubernetes.io/projected/d984c7b0-936c-4728-8cbd-be31e836a65b-kube-api-access-7nxqw\") pod \"opendatahub-operator-controller-manager-8bbd955f5-c8cm4\" (UID: \"d984c7b0-936c-4728-8cbd-be31e836a65b\") " pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.605433 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.605396 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:50.780395 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:50.780372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4"] Jul 08 01:48:50.783131 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:48:50.783096 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd984c7b0_936c_4728_8cbd_be31e836a65b.slice/crio-9b3bf409a34fe00e7bce5c2d88c7ebc1cc1d33ae47903781dd10f75b9c6d462a WatchSource:0}: Error finding container 9b3bf409a34fe00e7bce5c2d88c7ebc1cc1d33ae47903781dd10f75b9c6d462a: Status 404 returned error can't find the container with id 9b3bf409a34fe00e7bce5c2d88c7ebc1cc1d33ae47903781dd10f75b9c6d462a Jul 08 01:48:51.616422 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:51.616322 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" event={"ID":"d984c7b0-936c-4728-8cbd-be31e836a65b","Type":"ContainerStarted","Data":"9b3bf409a34fe00e7bce5c2d88c7ebc1cc1d33ae47903781dd10f75b9c6d462a"} Jul 08 01:48:54.631190 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:54.631158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" event={"ID":"d984c7b0-936c-4728-8cbd-be31e836a65b","Type":"ContainerStarted","Data":"39d432128e24301cbef04314cf0c23ba2a3fcc9f0224eb086c40a3b2d08111cf"} Jul 08 01:48:55.638185 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:55.638150 2560 generic.go:356] "Generic (PLEG): container finished" podID="d984c7b0-936c-4728-8cbd-be31e836a65b" containerID="39d432128e24301cbef04314cf0c23ba2a3fcc9f0224eb086c40a3b2d08111cf" exitCode=0 Jul 08 01:48:55.638623 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:55.638234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" event={"ID":"d984c7b0-936c-4728-8cbd-be31e836a65b","Type":"ContainerDied","Data":"39d432128e24301cbef04314cf0c23ba2a3fcc9f0224eb086c40a3b2d08111cf"} Jul 08 01:48:56.644014 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:56.643978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" event={"ID":"d984c7b0-936c-4728-8cbd-be31e836a65b","Type":"ContainerStarted","Data":"b00b205ea9a4020174512e08611de4755cc55fbe630fa2094af65207f085aec7"} Jul 08 01:48:56.644462 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:56.644112 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:48:56.666804 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:48:56.666757 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" podStartSLOduration=2.228216739 podStartE2EDuration="6.666740625s" totalImagesPullingTime="3.377384602s" totalInitContainerRuntime="1.061139284s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:48:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:48:56.665227394 +0000 UTC m=+368.119783020" watchObservedRunningTime="2026-07-08 01:48:56.666740625 +0000 UTC m=+368.121296269" Jul 08 01:49:07.649984 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:07.649952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8bbd955f5-c8cm4" Jul 08 01:49:10.264867 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.264816 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4pmpx"] Jul 08 01:49:10.274413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.274386 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.277900 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.277869 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 01:49:10.278033 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.277899 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qf6ng\"" Jul 08 01:49:10.281688 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.281653 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4pmpx"] Jul 08 01:49:10.383653 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.383624 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs"] Jul 08 01:49:10.389409 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.389378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.392499 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.392475 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 01:49:10.392621 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.392523 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 01:49:10.392688 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.392638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bq7s5\"" Jul 08 01:49:10.392745 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.392714 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 01:49:10.401747 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.401726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs"] Jul 08 01:49:10.425799 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.425768 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs769\" (UniqueName: \"kubernetes.io/projected/624e05de-7460-4e39-8ecd-549a27d15e40-kube-api-access-gs769\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.425799 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.425802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.526943 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.526853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs769\" (UniqueName: \"kubernetes.io/projected/624e05de-7460-4e39-8ecd-549a27d15e40-kube-api-access-gs769\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.526943 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.526892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.526943 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.526941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e885f47e-16ba-4698-8284-e285b8d4ac67-manager-config\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.527204 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.526963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.527204 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.526987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8s47\" (UniqueName: \"kubernetes.io/projected/e885f47e-16ba-4698-8284-e285b8d4ac67-kube-api-access-f8s47\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.527204 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.527014 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-metrics-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.527204 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:49:10.527038 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 01:49:10.527204 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:49:10.527108 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert podName:624e05de-7460-4e39-8ecd-549a27d15e40 nodeName:}" failed. No retries permitted until 2026-07-08 01:49:11.027090156 +0000 UTC m=+382.481645780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert") pod "odh-model-controller-7998c67dc4-4pmpx" (UID: "624e05de-7460-4e39-8ecd-549a27d15e40") : secret "odh-model-controller-webhook-cert" not found Jul 08 01:49:10.537952 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.537930 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs769\" (UniqueName: \"kubernetes.io/projected/624e05de-7460-4e39-8ecd-549a27d15e40-kube-api-access-gs769\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:10.628342 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.628310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e885f47e-16ba-4698-8284-e285b8d4ac67-manager-config\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.628342 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.628342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.628582 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.628359 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8s47\" (UniqueName: \"kubernetes.io/projected/e885f47e-16ba-4698-8284-e285b8d4ac67-kube-api-access-f8s47\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.628582 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.628479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-metrics-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.629082 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.629051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e885f47e-16ba-4698-8284-e285b8d4ac67-manager-config\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.630878 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.630859 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-metrics-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.631034 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.631015 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e885f47e-16ba-4698-8284-e285b8d4ac67-cert\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.636281 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.636257 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8s47\" (UniqueName: \"kubernetes.io/projected/e885f47e-16ba-4698-8284-e285b8d4ac67-kube-api-access-f8s47\") pod \"lws-controller-manager-7769fc489f-qb5gs\" (UID: \"e885f47e-16ba-4698-8284-e285b8d4ac67\") " pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.699329 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.699282 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:10.837124 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:10.837094 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs"] Jul 08 01:49:10.840164 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:49:10.840133 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode885f47e_16ba_4698_8284_e285b8d4ac67.slice/crio-5f61286841a5edd8185935427f970f8a6031600eadecebfcef41ea98e4555fa3 WatchSource:0}: Error finding container 5f61286841a5edd8185935427f970f8a6031600eadecebfcef41ea98e4555fa3: Status 404 returned error can't find the container with id 5f61286841a5edd8185935427f970f8a6031600eadecebfcef41ea98e4555fa3 Jul 08 01:49:11.032191 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.032158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:11.034507 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.034441 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/624e05de-7460-4e39-8ecd-549a27d15e40-cert\") pod \"odh-model-controller-7998c67dc4-4pmpx\" (UID: \"624e05de-7460-4e39-8ecd-549a27d15e40\") " pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:11.185176 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.185139 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:11.315991 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.315912 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4pmpx"] Jul 08 01:49:11.318486 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:49:11.318461 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624e05de_7460_4e39_8ecd_549a27d15e40.slice/crio-3ac60738597162c503f6912bb1bbaa729844fb50c157fd82ea3f9c176148e9ae WatchSource:0}: Error finding container 3ac60738597162c503f6912bb1bbaa729844fb50c157fd82ea3f9c176148e9ae: Status 404 returned error can't find the container with id 3ac60738597162c503f6912bb1bbaa729844fb50c157fd82ea3f9c176148e9ae Jul 08 01:49:11.708289 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.708200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" event={"ID":"624e05de-7460-4e39-8ecd-549a27d15e40","Type":"ContainerStarted","Data":"3ac60738597162c503f6912bb1bbaa729844fb50c157fd82ea3f9c176148e9ae"} Jul 08 01:49:11.709378 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:11.709351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" event={"ID":"e885f47e-16ba-4698-8284-e285b8d4ac67","Type":"ContainerStarted","Data":"5f61286841a5edd8185935427f970f8a6031600eadecebfcef41ea98e4555fa3"} Jul 08 01:49:13.721421 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:13.721386 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" event={"ID":"e885f47e-16ba-4698-8284-e285b8d4ac67","Type":"ContainerStarted","Data":"ab5b108e26de10f23829e3897c1c56442237f761dc3457ae2e5b546bfb1fa2ad"} Jul 08 01:49:13.721913 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:13.721525 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:13.739292 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:13.739229 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" podStartSLOduration=1.899587538 podStartE2EDuration="3.739211057s" totalImagesPullingTime="1.839623519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:49:13.737849673 +0000 UTC m=+385.192405319" watchObservedRunningTime="2026-07-08 01:49:13.739211057 +0000 UTC m=+385.193766702" Jul 08 01:49:14.728217 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:14.728184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" event={"ID":"624e05de-7460-4e39-8ecd-549a27d15e40","Type":"ContainerStarted","Data":"a05493391a4fc39b8ba4ebbef35e37bd0d7ffe8d2b408b2bf5acb2ece245d075"} Jul 08 01:49:14.728638 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:14.728386 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:14.744256 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:14.744204 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" podStartSLOduration=1.4275142060000001 podStartE2EDuration="4.744188203s" totalImagesPullingTime="3.316673997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:49:14.742471215 +0000 UTC m=+386.197026861" watchObservedRunningTime="2026-07-08 01:49:14.744188203 +0000 UTC m=+386.198743848" Jul 08 01:49:18.540462 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.540424 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh"] Jul 08 01:49:18.543656 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.543635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.546479 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.546458 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 01:49:18.546780 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.546762 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 01:49:18.546910 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.546808 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 01:49:18.546910 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.546871 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 01:49:18.547046 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.547022 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t7frg\"" Jul 08 01:49:18.547108 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.547056 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 01:49:18.547108 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.547073 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 01:49:18.556415 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.556394 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh"] Jul 08 01:49:18.706975 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.706939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707145 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.706992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707145 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.707017 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707145 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.707065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd70d71-741b-482d-bc0d-a896bfe77652-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707145 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.707085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56m59\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-kube-api-access-56m59\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707145 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.707123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.707308 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.707148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.807760 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.807733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.807949 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.807779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.807949 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.807800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.807949 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.807823 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd70d71-741b-482d-bc0d-a896bfe77652-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.808050 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.807997 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56m59\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-kube-api-access-56m59\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.808050 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.808040 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.808119 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.808074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.808469 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.808440 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.810330 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.810300 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.810445 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.810381 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.810550 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.810533 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd70d71-741b-482d-bc0d-a896bfe77652-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.810637 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.810619 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acd70d71-741b-482d-bc0d-a896bfe77652-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.825085 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.825064 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56m59\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-kube-api-access-56m59\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.826994 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.826970 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd70d71-741b-482d-bc0d-a896bfe77652-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-v6cjh\" (UID: \"acd70d71-741b-482d-bc0d-a896bfe77652\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.852932 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.852907 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:18.994412 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:18.994388 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh"] Jul 08 01:49:18.996632 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:49:18.996600 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd70d71_741b_482d_bc0d_a896bfe77652.slice/crio-d3f0adf0366ca4aef0042ccf660e8a5b452d1fe920855f34f183702e12373dba WatchSource:0}: Error finding container d3f0adf0366ca4aef0042ccf660e8a5b452d1fe920855f34f183702e12373dba: Status 404 returned error can't find the container with id d3f0adf0366ca4aef0042ccf660e8a5b452d1fe920855f34f183702e12373dba Jul 08 01:49:19.754487 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:19.754446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" event={"ID":"acd70d71-741b-482d-bc0d-a896bfe77652","Type":"ContainerStarted","Data":"d3f0adf0366ca4aef0042ccf660e8a5b452d1fe920855f34f183702e12373dba"} Jul 08 01:49:21.452712 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:21.452660 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Jul 08 01:49:21.453020 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:21.452744 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Jul 08 01:49:21.766564 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:21.766473 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" event={"ID":"acd70d71-741b-482d-bc0d-a896bfe77652","Type":"ContainerStarted","Data":"5944ec990c330ad64c0ab9e8581af78be9e7e001642e6ea6c5aa94b2581ab70a"} Jul 08 01:49:21.766710 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:21.766588 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:49:21.800156 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:21.800088 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" podStartSLOduration=1.3463699199999999 podStartE2EDuration="3.800069076s" totalImagesPullingTime="2.453699156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:49:21.79918037 +0000 UTC m=+393.253736014" watchObservedRunningTime="2026-07-08 01:49:21.800069076 +0000 UTC m=+393.254624723" Jul 08 01:49:22.774887 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:22.774783 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-v6cjh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 01:49:22.775294 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:22.774936 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" podUID="acd70d71-741b-482d-bc0d-a896bfe77652" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 01:49:24.730622 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:24.730591 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7769fc489f-qb5gs" Jul 08 01:49:25.734583 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:25.734557 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-4pmpx" Jul 08 01:49:25.773784 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:49:25.773751 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-v6cjh" Jul 08 01:50:26.782813 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.782736 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb"] Jul 08 01:50:26.786165 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.786149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.788513 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.788489 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-27b8d\"" Jul 08 01:50:26.788633 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.788547 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 01:50:26.788633 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.788552 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 01:50:26.794644 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.794621 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb"] Jul 08 01:50:26.877202 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.877163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.877202 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.877205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tm2z\" (UniqueName: \"kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.877420 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.877290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.977798 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.977760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tm2z\" (UniqueName: \"kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.977946 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.977823 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.978004 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.977982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.978217 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.978202 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.978321 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.978304 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:26.986570 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:26.986538 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tm2z\" (UniqueName: \"kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:27.095694 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.095653 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:27.179492 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.179435 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p"] Jul 08 01:50:27.184175 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.184150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.190888 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.190806 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p"] Jul 08 01:50:27.233136 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.233107 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb"] Jul 08 01:50:27.236289 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:50:27.236249 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acb6b09_2df9_4c29_9990_569c83b5997f.slice/crio-62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809 WatchSource:0}: Error finding container 62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809: Status 404 returned error can't find the container with id 62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809 Jul 08 01:50:27.280006 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.279971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.280142 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.280040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.280142 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.280084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km5mq\" (UniqueName: \"kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.381365 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.381282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.381365 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.381328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km5mq\" (UniqueName: \"kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.381605 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.381412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.381666 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.381638 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.381746 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.381725 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.389549 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.389524 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km5mq\" (UniqueName: \"kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.497051 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.496997 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:27.588092 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.588058 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m"] Jul 08 01:50:27.598657 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.598625 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.603853 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.603801 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m"] Jul 08 01:50:27.648510 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.648487 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p"] Jul 08 01:50:27.650690 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:50:27.650655 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31220df7_bcde_48b4_a1b4_1a61ffb749c6.slice/crio-8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42 WatchSource:0}: Error finding container 8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42: Status 404 returned error can't find the container with id 8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42 Jul 08 01:50:27.683789 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.683763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrv25\" (UniqueName: \"kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.683923 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.683820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.683923 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.683893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.784747 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.784713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.785116 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.784785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrv25\" (UniqueName: \"kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.785116 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.784861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.785116 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.785107 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.785236 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.785184 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.793303 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.793278 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrv25\" (UniqueName: \"kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.911324 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.911233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:27.976867 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.976821 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p"] Jul 08 01:50:27.980562 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.980535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:27.985826 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.985793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:27.985982 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.985915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6589m\" (UniqueName: \"kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:27.986041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.985982 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:27.987987 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:27.987965 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p"] Jul 08 01:50:28.047455 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.047423 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m"] Jul 08 01:50:28.049940 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:50:28.049912 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559b81e8_b609_403f_9e0a_cecf99b401ca.slice/crio-0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f WatchSource:0}: Error finding container 0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f: Status 404 returned error can't find the container with id 0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f Jul 08 01:50:28.072375 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.072344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" event={"ID":"559b81e8-b609-403f-9e0a-cecf99b401ca","Type":"ContainerStarted","Data":"0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f"} Jul 08 01:50:28.074013 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.073990 2560 generic.go:356] "Generic (PLEG): container finished" podID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerID="19d184864d9274192048fac63da5ef230df0f53eb5fe41955174aeb3d7c2ee87" exitCode=0 Jul 08 01:50:28.074111 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.074065 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" event={"ID":"31220df7-bcde-48b4-a1b4-1a61ffb749c6","Type":"ContainerDied","Data":"19d184864d9274192048fac63da5ef230df0f53eb5fe41955174aeb3d7c2ee87"} Jul 08 01:50:28.074111 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.074102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" event={"ID":"31220df7-bcde-48b4-a1b4-1a61ffb749c6","Type":"ContainerStarted","Data":"8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42"} Jul 08 01:50:28.075524 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.075501 2560 generic.go:356] "Generic (PLEG): container finished" podID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerID="9d9063123c3ae1127ab39c3f0f6649d3c44d73b0568fc96bdad5c6004c2a5457" exitCode=0 Jul 08 01:50:28.075599 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.075547 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" event={"ID":"3acb6b09-2df9-4c29-9990-569c83b5997f","Type":"ContainerDied","Data":"9d9063123c3ae1127ab39c3f0f6649d3c44d73b0568fc96bdad5c6004c2a5457"} Jul 08 01:50:28.075599 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.075580 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" event={"ID":"3acb6b09-2df9-4c29-9990-569c83b5997f","Type":"ContainerStarted","Data":"62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809"} Jul 08 01:50:28.087143 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.087119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.087280 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.087174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.087554 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.087454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6589m\" (UniqueName: \"kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.087650 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.087550 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.087650 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.087602 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.097814 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.097787 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6589m\" (UniqueName: \"kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.306291 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.306255 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:28.438648 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:28.438625 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p"] Jul 08 01:50:28.441002 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:50:28.440980 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd545210_0ccc_4eef_b584_d1c6aade3ae2.slice/crio-a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64 WatchSource:0}: Error finding container a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64: Status 404 returned error can't find the container with id a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64 Jul 08 01:50:29.083853 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:29.083774 2560 generic.go:356] "Generic (PLEG): container finished" podID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerID="2ed0f760351c775033de81b8347d6f34489a158b3233a7e3ede0cf5d52cc36ea" exitCode=0 Jul 08 01:50:29.084304 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:29.083877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" event={"ID":"fd545210-0ccc-4eef-b584-d1c6aade3ae2","Type":"ContainerDied","Data":"2ed0f760351c775033de81b8347d6f34489a158b3233a7e3ede0cf5d52cc36ea"} Jul 08 01:50:29.084304 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:29.083940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" event={"ID":"fd545210-0ccc-4eef-b584-d1c6aade3ae2","Type":"ContainerStarted","Data":"a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64"} Jul 08 01:50:29.086014 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:29.085992 2560 generic.go:356] "Generic (PLEG): container finished" podID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerID="5f952c11c2b9d5c59083ee952149fbce6f7384e1bc6feb984549afa6601b3a0b" exitCode=0 Jul 08 01:50:29.086151 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:29.086031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" event={"ID":"559b81e8-b609-403f-9e0a-cecf99b401ca","Type":"ContainerDied","Data":"5f952c11c2b9d5c59083ee952149fbce6f7384e1bc6feb984549afa6601b3a0b"} Jul 08 01:50:30.093167 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.093131 2560 generic.go:356] "Generic (PLEG): container finished" podID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerID="0f4223d58ef2b0f609d114a985feae75d1d96dbcc22425f06e87a767dc78850a" exitCode=0 Jul 08 01:50:30.093647 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.093229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" event={"ID":"559b81e8-b609-403f-9e0a-cecf99b401ca","Type":"ContainerDied","Data":"0f4223d58ef2b0f609d114a985feae75d1d96dbcc22425f06e87a767dc78850a"} Jul 08 01:50:30.095201 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.095179 2560 generic.go:356] "Generic (PLEG): container finished" podID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerID="30ae31c0e52ff5d30962c2f90a28600b823869571aa427e04727bd8af47f6590" exitCode=0 Jul 08 01:50:30.095269 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.095249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" event={"ID":"31220df7-bcde-48b4-a1b4-1a61ffb749c6","Type":"ContainerDied","Data":"30ae31c0e52ff5d30962c2f90a28600b823869571aa427e04727bd8af47f6590"} Jul 08 01:50:30.097416 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.097395 2560 generic.go:356] "Generic (PLEG): container finished" podID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerID="60caaa7f97cbe2eb71b380f9871abb1d75ab483d1a49052fc81201bb19673fb9" exitCode=0 Jul 08 01:50:30.097522 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:30.097420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" event={"ID":"3acb6b09-2df9-4c29-9990-569c83b5997f","Type":"ContainerDied","Data":"60caaa7f97cbe2eb71b380f9871abb1d75ab483d1a49052fc81201bb19673fb9"} Jul 08 01:50:31.104016 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.103981 2560 generic.go:356] "Generic (PLEG): container finished" podID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerID="37d8686205f469d8f6ce9c2d75fa22a8945905113617f8af33bd41d01cef41ef" exitCode=0 Jul 08 01:50:31.104460 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.104068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" event={"ID":"559b81e8-b609-403f-9e0a-cecf99b401ca","Type":"ContainerDied","Data":"37d8686205f469d8f6ce9c2d75fa22a8945905113617f8af33bd41d01cef41ef"} Jul 08 01:50:31.106019 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.105995 2560 generic.go:356] "Generic (PLEG): container finished" podID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerID="36cf6e1580cf7e5fdd44f0f990721e0dfb194b887ad64a9fbb521c9344469b73" exitCode=0 Jul 08 01:50:31.106106 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.106086 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" event={"ID":"31220df7-bcde-48b4-a1b4-1a61ffb749c6","Type":"ContainerDied","Data":"36cf6e1580cf7e5fdd44f0f990721e0dfb194b887ad64a9fbb521c9344469b73"} Jul 08 01:50:31.108175 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.108155 2560 generic.go:356] "Generic (PLEG): container finished" podID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerID="2785962f77d30c8d6b1d777b79dff9fcdaaa3f189d4f1444005598ed6b4b7580" exitCode=0 Jul 08 01:50:31.108286 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.108216 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" event={"ID":"3acb6b09-2df9-4c29-9990-569c83b5997f","Type":"ContainerDied","Data":"2785962f77d30c8d6b1d777b79dff9fcdaaa3f189d4f1444005598ed6b4b7580"} Jul 08 01:50:31.109811 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.109787 2560 generic.go:356] "Generic (PLEG): container finished" podID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerID="d1ee824305e7de355031812701c95283725a6b915054a12c91e0657d3aedf2a7" exitCode=0 Jul 08 01:50:31.109916 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:31.109812 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" event={"ID":"fd545210-0ccc-4eef-b584-d1c6aade3ae2","Type":"ContainerDied","Data":"d1ee824305e7de355031812701c95283725a6b915054a12c91e0657d3aedf2a7"} Jul 08 01:50:32.116710 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.116671 2560 generic.go:356] "Generic (PLEG): container finished" podID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerID="20e6c1ba591bdffcbd7959f04c9b2badc8d70f1854400cc51a852814ee873042" exitCode=0 Jul 08 01:50:32.117253 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.116757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" event={"ID":"fd545210-0ccc-4eef-b584-d1c6aade3ae2","Type":"ContainerDied","Data":"20e6c1ba591bdffcbd7959f04c9b2badc8d70f1854400cc51a852814ee873042"} Jul 08 01:50:32.264025 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.263997 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:32.298175 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.298152 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:32.300962 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.300943 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:32.324543 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324519 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util\") pod \"559b81e8-b609-403f-9e0a-cecf99b401ca\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " Jul 08 01:50:32.324670 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324572 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util\") pod \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " Jul 08 01:50:32.324670 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324592 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle\") pod \"3acb6b09-2df9-4c29-9990-569c83b5997f\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " Jul 08 01:50:32.324670 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324619 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tm2z\" (UniqueName: \"kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z\") pod \"3acb6b09-2df9-4c29-9990-569c83b5997f\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " Jul 08 01:50:32.324670 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324650 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km5mq\" (UniqueName: \"kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq\") pod \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " Jul 08 01:50:32.324911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324684 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle\") pod \"559b81e8-b609-403f-9e0a-cecf99b401ca\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " Jul 08 01:50:32.324911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324718 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrv25\" (UniqueName: \"kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25\") pod \"559b81e8-b609-403f-9e0a-cecf99b401ca\" (UID: \"559b81e8-b609-403f-9e0a-cecf99b401ca\") " Jul 08 01:50:32.324911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324766 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle\") pod \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\" (UID: \"31220df7-bcde-48b4-a1b4-1a61ffb749c6\") " Jul 08 01:50:32.324911 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.324807 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util\") pod \"3acb6b09-2df9-4c29-9990-569c83b5997f\" (UID: \"3acb6b09-2df9-4c29-9990-569c83b5997f\") " Jul 08 01:50:32.325493 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.325340 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle" (OuterVolumeSpecName: "bundle") pod "3acb6b09-2df9-4c29-9990-569c83b5997f" (UID: "3acb6b09-2df9-4c29-9990-569c83b5997f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.326117 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.325849 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle" (OuterVolumeSpecName: "bundle") pod "31220df7-bcde-48b4-a1b4-1a61ffb749c6" (UID: "31220df7-bcde-48b4-a1b4-1a61ffb749c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.326339 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.326296 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle" (OuterVolumeSpecName: "bundle") pod "559b81e8-b609-403f-9e0a-cecf99b401ca" (UID: "559b81e8-b609-403f-9e0a-cecf99b401ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.327334 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.327296 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z" (OuterVolumeSpecName: "kube-api-access-6tm2z") pod "3acb6b09-2df9-4c29-9990-569c83b5997f" (UID: "3acb6b09-2df9-4c29-9990-569c83b5997f"). InnerVolumeSpecName "kube-api-access-6tm2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:50:32.327628 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.327598 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq" (OuterVolumeSpecName: "kube-api-access-km5mq") pod "31220df7-bcde-48b4-a1b4-1a61ffb749c6" (UID: "31220df7-bcde-48b4-a1b4-1a61ffb749c6"). InnerVolumeSpecName "kube-api-access-km5mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:50:32.328093 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.328071 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25" (OuterVolumeSpecName: "kube-api-access-lrv25") pod "559b81e8-b609-403f-9e0a-cecf99b401ca" (UID: "559b81e8-b609-403f-9e0a-cecf99b401ca"). InnerVolumeSpecName "kube-api-access-lrv25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:50:32.331183 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.331161 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util" (OuterVolumeSpecName: "util") pod "3acb6b09-2df9-4c29-9990-569c83b5997f" (UID: "3acb6b09-2df9-4c29-9990-569c83b5997f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.331639 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.331616 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util" (OuterVolumeSpecName: "util") pod "31220df7-bcde-48b4-a1b4-1a61ffb749c6" (UID: "31220df7-bcde-48b4-a1b4-1a61ffb749c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.334041 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.334013 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util" (OuterVolumeSpecName: "util") pod "559b81e8-b609-403f-9e0a-cecf99b401ca" (UID: "559b81e8-b609-403f-9e0a-cecf99b401ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425552 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425605 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrv25\" (UniqueName: \"kubernetes.io/projected/559b81e8-b609-403f-9e0a-cecf99b401ca-kube-api-access-lrv25\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425621 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425632 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425645 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/559b81e8-b609-403f-9e0a-cecf99b401ca-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425656 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31220df7-bcde-48b4-a1b4-1a61ffb749c6-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425668 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb6b09-2df9-4c29-9990-569c83b5997f-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.425671 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425679 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tm2z\" (UniqueName: \"kubernetes.io/projected/3acb6b09-2df9-4c29-9990-569c83b5997f-kube-api-access-6tm2z\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:32.426065 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:32.425692 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km5mq\" (UniqueName: \"kubernetes.io/projected/31220df7-bcde-48b4-a1b4-1a61ffb749c6-kube-api-access-km5mq\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:33.123813 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.123782 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" Jul 08 01:50:33.123813 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.123795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb" event={"ID":"3acb6b09-2df9-4c29-9990-569c83b5997f","Type":"ContainerDied","Data":"62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809"} Jul 08 01:50:33.124266 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.123866 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62c968a04d9de449cf65e61d1096d0c52d5d40e079a1becc09f3fb535d2d8809" Jul 08 01:50:33.125818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.125786 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" Jul 08 01:50:33.125818 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.125809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m" event={"ID":"559b81e8-b609-403f-9e0a-cecf99b401ca","Type":"ContainerDied","Data":"0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f"} Jul 08 01:50:33.126105 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.125876 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e893e0732932e6eedf737d4d7e8e25681ef8469110828d585ea4ed5d3edf78f" Jul 08 01:50:33.128016 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.127989 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" event={"ID":"31220df7-bcde-48b4-a1b4-1a61ffb749c6","Type":"ContainerDied","Data":"8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42"} Jul 08 01:50:33.128100 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.128026 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b0b6869277939d9e344359b4e25193ba171efa6ede594bc48d31e3ad4ea7e42" Jul 08 01:50:33.128100 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.128079 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p" Jul 08 01:50:33.259425 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.259402 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:33.332593 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.332557 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util\") pod \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " Jul 08 01:50:33.332593 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.332596 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle\") pod \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " Jul 08 01:50:33.332866 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.332630 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6589m\" (UniqueName: \"kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m\") pod \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\" (UID: \"fd545210-0ccc-4eef-b584-d1c6aade3ae2\") " Jul 08 01:50:33.333108 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.333077 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle" (OuterVolumeSpecName: "bundle") pod "fd545210-0ccc-4eef-b584-d1c6aade3ae2" (UID: "fd545210-0ccc-4eef-b584-d1c6aade3ae2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:33.334654 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.334631 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m" (OuterVolumeSpecName: "kube-api-access-6589m") pod "fd545210-0ccc-4eef-b584-d1c6aade3ae2" (UID: "fd545210-0ccc-4eef-b584-d1c6aade3ae2"). InnerVolumeSpecName "kube-api-access-6589m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:50:33.338381 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.338360 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util" (OuterVolumeSpecName: "util") pod "fd545210-0ccc-4eef-b584-d1c6aade3ae2" (UID: "fd545210-0ccc-4eef-b584-d1c6aade3ae2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:50:33.433279 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.433236 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-bundle\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:33.433279 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.433271 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6589m\" (UniqueName: \"kubernetes.io/projected/fd545210-0ccc-4eef-b584-d1c6aade3ae2-kube-api-access-6589m\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:33.433279 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:33.433284 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd545210-0ccc-4eef-b584-d1c6aade3ae2-util\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:50:34.135216 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:34.135186 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" Jul 08 01:50:34.135657 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:34.135221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p" event={"ID":"fd545210-0ccc-4eef-b584-d1c6aade3ae2","Type":"ContainerDied","Data":"a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64"} Jul 08 01:50:34.135657 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:50:34.135269 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2fc8f6aaa5ed5afc1654d840cff5c1add2d3d8368c366ec417bf437af1a6e64" Jul 08 01:51:11.636396 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636352 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp"] Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636851 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636872 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636893 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636903 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636913 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636921 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636928 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636935 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636947 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636954 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636964 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636971 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.636991 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637034 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="util" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637044 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637052 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637086 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637094 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637103 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637111 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637125 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637151 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="pull" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637159 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637166 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637279 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3acb6b09-2df9-4c29-9990-569c83b5997f" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637289 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd545210-0ccc-4eef-b584-d1c6aade3ae2" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637305 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="559b81e8-b609-403f-9e0a-cecf99b401ca" containerName="extract" Jul 08 01:51:11.637298 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.637314 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="31220df7-bcde-48b4-a1b4-1a61ffb749c6" containerName="extract" Jul 08 01:51:11.645698 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.645673 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.648756 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.648731 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 01:51:11.648898 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.648768 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 01:51:11.648898 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.648861 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dhp5g\"" Jul 08 01:51:11.657398 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.657372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp"] Jul 08 01:51:11.769647 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.769606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.769647 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.769649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgwdp\" (UniqueName: \"kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.870920 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.870882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.870920 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.870925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgwdp\" (UniqueName: \"kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.871332 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.871308 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.890739 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.890669 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgwdp\" (UniqueName: \"kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:11.956596 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:11.956550 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:12.095720 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.095695 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp"] Jul 08 01:51:12.100289 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:51:12.098950 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9533afd6_1959_45a7_b21a_339091b71b6f.slice/crio-21b45270ec9b66d16515d7e90a8eeb0f69e2d6158bf5c9b0c121fe039838b84c WatchSource:0}: Error finding container 21b45270ec9b66d16515d7e90a8eeb0f69e2d6158bf5c9b0c121fe039838b84c: Status 404 returned error can't find the container with id 21b45270ec9b66d16515d7e90a8eeb0f69e2d6158bf5c9b0c121fe039838b84c Jul 08 01:51:12.315175 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.315137 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" event={"ID":"9533afd6-1959-45a7-b21a-339091b71b6f","Type":"ContainerStarted","Data":"21b45270ec9b66d16515d7e90a8eeb0f69e2d6158bf5c9b0c121fe039838b84c"} Jul 08 01:51:12.381191 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.381141 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp"] Jul 08 01:51:12.385354 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.385324 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp"] Jul 08 01:51:12.414765 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.414732 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:12.417964 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.417946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.439642 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.439605 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:12.577681 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.577596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jvt7\" (UniqueName: \"kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.577681 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.577648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.678674 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.678636 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.679149 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.678754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jvt7\" (UniqueName: \"kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.679149 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.679005 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.689560 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.689533 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jvt7\" (UniqueName: \"kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jcxs9\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:12.728635 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.728603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:13.000194 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:12.999867 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:13.030675 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:51:13.030640 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf956fc6b_25aa_48ff_b89a_ca20d7e6992e.slice/crio-4003f9d716744ab1a74cc641b4f5710015497bf62bdadbfe2dadbbdf3d80469e WatchSource:0}: Error finding container 4003f9d716744ab1a74cc641b4f5710015497bf62bdadbfe2dadbbdf3d80469e: Status 404 returned error can't find the container with id 4003f9d716744ab1a74cc641b4f5710015497bf62bdadbfe2dadbbdf3d80469e Jul 08 01:51:13.322750 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:13.322718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" event={"ID":"f956fc6b-25aa-48ff-b89a-ca20d7e6992e","Type":"ContainerStarted","Data":"4003f9d716744ab1a74cc641b4f5710015497bf62bdadbfe2dadbbdf3d80469e"} Jul 08 01:51:18.355140 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.355088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" event={"ID":"f956fc6b-25aa-48ff-b89a-ca20d7e6992e","Type":"ContainerStarted","Data":"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d"} Jul 08 01:51:18.355545 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.355165 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:18.356934 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.356899 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" containerName="manager" containerID="cri-o://af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561" gracePeriod=2 Jul 08 01:51:18.382399 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.382347 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" podStartSLOduration=2.12845869 podStartE2EDuration="6.382330207s" totalImagesPullingTime="4.253871517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:51:18.380383791 +0000 UTC m=+509.834939436" watchObservedRunningTime="2026-07-08 01:51:18.382330207 +0000 UTC m=+509.836885922" Jul 08 01:51:18.600695 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.600668 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:18.602972 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:51:18.602935 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" is forbidden: User \"system:node:ip-10-0-136-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-194.ec2.internal' and this object" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:18.630405 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.630322 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume\") pod \"9533afd6-1959-45a7-b21a-339091b71b6f\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " Jul 08 01:51:18.630405 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.630363 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgwdp\" (UniqueName: \"kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp\") pod \"9533afd6-1959-45a7-b21a-339091b71b6f\" (UID: \"9533afd6-1959-45a7-b21a-339091b71b6f\") " Jul 08 01:51:18.630664 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.630638 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9533afd6-1959-45a7-b21a-339091b71b6f" (UID: "9533afd6-1959-45a7-b21a-339091b71b6f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:51:18.632455 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.632421 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp" (OuterVolumeSpecName: "kube-api-access-lgwdp") pod "9533afd6-1959-45a7-b21a-339091b71b6f" (UID: "9533afd6-1959-45a7-b21a-339091b71b6f"). InnerVolumeSpecName "kube-api-access-lgwdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:51:18.730980 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.730943 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9533afd6-1959-45a7-b21a-339091b71b6f-extensions-socket-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:51:18.730980 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:18.730972 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgwdp\" (UniqueName: \"kubernetes.io/projected/9533afd6-1959-45a7-b21a-339091b71b6f-kube-api-access-lgwdp\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:51:19.028798 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:51:19.028705 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" is forbidden: User \"system:node:ip-10-0-136-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-194.ec2.internal' and this object" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:19.030115 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.030091 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" path="/var/lib/kubelet/pods/9533afd6-1959-45a7-b21a-339091b71b6f/volumes" Jul 08 01:51:19.363499 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.363462 2560 generic.go:356] "Generic (PLEG): container finished" podID="9533afd6-1959-45a7-b21a-339091b71b6f" containerID="af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561" exitCode=2 Jul 08 01:51:19.363976 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.363509 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:19.363976 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.363575 2560 scope.go:117] "RemoveContainer" containerID="af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561" Jul 08 01:51:19.375956 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:51:19.368275 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lzcnp\" is forbidden: User \"system:node:ip-10-0-136-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-194.ec2.internal' and this object" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lzcnp" Jul 08 01:51:19.394039 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.394015 2560 scope.go:117] "RemoveContainer" containerID="af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561" Jul 08 01:51:19.394389 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:51:19.394369 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561\": container with ID starting with af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561 not found: ID does not exist" containerID="af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561" Jul 08 01:51:19.394435 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:19.394401 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561"} err="failed to get container status \"af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561\": rpc error: code = NotFound desc = could not find container \"af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561\": container with ID starting with af0a530dc9261c2d3a54709601935d1972694946196da1f7296c5c22ed711561 not found: ID does not exist" Jul 08 01:51:29.366500 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.366468 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:29.707428 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.707395 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 01:51:29.707710 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.707698 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" containerName="manager" Jul 08 01:51:29.707753 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.707711 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" containerName="manager" Jul 08 01:51:29.707803 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.707793 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9533afd6-1959-45a7-b21a-339091b71b6f" containerName="manager" Jul 08 01:51:29.712084 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.712065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.726902 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.726871 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 01:51:29.824444 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.824405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hc7\" (UniqueName: \"kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.824621 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.824468 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.924968 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.924868 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.924968 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.924970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hc7\" (UniqueName: \"kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.925370 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.925349 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:29.933196 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:29.933160 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hc7\" (UniqueName: \"kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7\") pod \"kuadrant-operator-controller-manager-86878c769f-x6hgv\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:30.022210 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.022170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:30.158610 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.158581 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 01:51:30.160892 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:51:30.160855 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff79c7e_5793_4fb6_b4ab_4395a6322cba.slice/crio-209687c52c1a1791e9cdb5ee168a06056d7f49301c8952834bf78b0bec70d719 WatchSource:0}: Error finding container 209687c52c1a1791e9cdb5ee168a06056d7f49301c8952834bf78b0bec70d719: Status 404 returned error can't find the container with id 209687c52c1a1791e9cdb5ee168a06056d7f49301c8952834bf78b0bec70d719 Jul 08 01:51:30.419293 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.419260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" event={"ID":"eff79c7e-5793-4fb6-b4ab-4395a6322cba","Type":"ContainerStarted","Data":"d7409ed12d658510d8b52568d778f97b72e36d9cd1243082d6a9ca624f718c78"} Jul 08 01:51:30.419695 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.419321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" event={"ID":"eff79c7e-5793-4fb6-b4ab-4395a6322cba","Type":"ContainerStarted","Data":"209687c52c1a1791e9cdb5ee168a06056d7f49301c8952834bf78b0bec70d719"} Jul 08 01:51:30.419695 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.419446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:30.440565 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:30.440466 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" podStartSLOduration=1.440449335 podStartE2EDuration="1.440449335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:51:30.438289022 +0000 UTC m=+521.892844666" watchObservedRunningTime="2026-07-08 01:51:30.440449335 +0000 UTC m=+521.895004980" Jul 08 01:51:41.427021 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.426943 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 01:51:41.497243 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.497213 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:41.497484 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.497459 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" podUID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" containerName="manager" containerID="cri-o://da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d" gracePeriod=10 Jul 08 01:51:41.746430 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.746398 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:41.831783 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.831739 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume\") pod \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " Jul 08 01:51:41.831990 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.831869 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jvt7\" (UniqueName: \"kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7\") pod \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\" (UID: \"f956fc6b-25aa-48ff-b89a-ca20d7e6992e\") " Jul 08 01:51:41.832282 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.832253 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f956fc6b-25aa-48ff-b89a-ca20d7e6992e" (UID: "f956fc6b-25aa-48ff-b89a-ca20d7e6992e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 01:51:41.834063 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.834037 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7" (OuterVolumeSpecName: "kube-api-access-8jvt7") pod "f956fc6b-25aa-48ff-b89a-ca20d7e6992e" (UID: "f956fc6b-25aa-48ff-b89a-ca20d7e6992e"). InnerVolumeSpecName "kube-api-access-8jvt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 01:51:41.933314 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.933274 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-extensions-socket-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:51:41.933314 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:41.933306 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jvt7\" (UniqueName: \"kubernetes.io/projected/f956fc6b-25aa-48ff-b89a-ca20d7e6992e-kube-api-access-8jvt7\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 01:51:42.484600 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.484563 2560 generic.go:356] "Generic (PLEG): container finished" podID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" containerID="da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d" exitCode=0 Jul 08 01:51:42.485128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.484627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" event={"ID":"f956fc6b-25aa-48ff-b89a-ca20d7e6992e","Type":"ContainerDied","Data":"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d"} Jul 08 01:51:42.485128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.484685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" event={"ID":"f956fc6b-25aa-48ff-b89a-ca20d7e6992e","Type":"ContainerDied","Data":"4003f9d716744ab1a74cc641b4f5710015497bf62bdadbfe2dadbbdf3d80469e"} Jul 08 01:51:42.485128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.484706 2560 scope.go:117] "RemoveContainer" containerID="da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d" Jul 08 01:51:42.485128 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.484634 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9" Jul 08 01:51:42.494495 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.494478 2560 scope.go:117] "RemoveContainer" containerID="da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d" Jul 08 01:51:42.494762 ip-10-0-136-194 kubenswrapper[2560]: E0708 01:51:42.494742 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d\": container with ID starting with da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d not found: ID does not exist" containerID="da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d" Jul 08 01:51:42.494799 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.494772 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d"} err="failed to get container status \"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d\": rpc error: code = NotFound desc = could not find container \"da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d\": container with ID starting with da68f2c2469c24b98a071106183cfd3b2f54cb94b1fc84ca3705e0a0ea61076d not found: ID does not exist" Jul 08 01:51:42.511650 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.511623 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:42.520413 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:42.520388 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jcxs9"] Jul 08 01:51:43.028684 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:51:43.028650 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" path="/var/lib/kubelet/pods/f956fc6b-25aa-48ff-b89a-ca20d7e6992e/volumes" Jul 08 01:53:36.616494 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.616419 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt"] Jul 08 01:53:36.618140 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.616773 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" containerName="manager" Jul 08 01:53:36.618140 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.616784 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" containerName="manager" Jul 08 01:53:36.618140 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.616849 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f956fc6b-25aa-48ff-b89a-ca20d7e6992e" containerName="manager" Jul 08 01:53:36.618772 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.618756 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.621314 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.621292 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hz9bp\"" Jul 08 01:53:36.621443 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.621316 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 01:53:36.621443 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.621296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 01:53:36.622148 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.622131 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 01:53:36.631024 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.631001 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt"] Jul 08 01:53:36.729320 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.729320 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729324 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823205e7-d413-4eff-85a0-6c854c7d6a7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.729577 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6qr\" (UniqueName: \"kubernetes.io/projected/823205e7-d413-4eff-85a0-6c854c7d6a7f-kube-api-access-cr6qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.729577 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729450 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.729577 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.729577 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.729505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830206 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823205e7-d413-4eff-85a0-6c854c7d6a7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830386 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6qr\" (UniqueName: \"kubernetes.io/projected/823205e7-d413-4eff-85a0-6c854c7d6a7f-kube-api-access-cr6qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830386 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830386 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830277 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830386 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830386 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830769 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830769 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830760 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.830907 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.830779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.832521 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.832503 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/823205e7-d413-4eff-85a0-6c854c7d6a7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.832795 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.832777 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/823205e7-d413-4eff-85a0-6c854c7d6a7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.839342 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.839322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6qr\" (UniqueName: \"kubernetes.io/projected/823205e7-d413-4eff-85a0-6c854c7d6a7f-kube-api-access-cr6qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kdqrt\" (UID: \"823205e7-d413-4eff-85a0-6c854c7d6a7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:36.929134 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:36.929049 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:37.064852 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:37.064814 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt"] Jul 08 01:53:37.067119 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:53:37.067094 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod823205e7_d413_4eff_85a0_6c854c7d6a7f.slice/crio-933b6052defc72b12f2e194ab3628214cd97a89aad57da8fba0e7a3ab6432ffb WatchSource:0}: Error finding container 933b6052defc72b12f2e194ab3628214cd97a89aad57da8fba0e7a3ab6432ffb: Status 404 returned error can't find the container with id 933b6052defc72b12f2e194ab3628214cd97a89aad57da8fba0e7a3ab6432ffb Jul 08 01:53:37.068457 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:37.068440 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 01:53:38.050637 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:38.050601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" event={"ID":"823205e7-d413-4eff-85a0-6c854c7d6a7f","Type":"ContainerStarted","Data":"933b6052defc72b12f2e194ab3628214cd97a89aad57da8fba0e7a3ab6432ffb"} Jul 08 01:53:39.057477 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:39.057444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" event={"ID":"823205e7-d413-4eff-85a0-6c854c7d6a7f","Type":"ContainerStarted","Data":"5bc93dda18ff12cfc1debf07cec6a9ede3ee913a71f4a2d87f62cea03ab14049"} Jul 08 01:53:39.079616 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:39.079553 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" podStartSLOduration=2.12550695 podStartE2EDuration="3.079532927s" totalImagesPullingTime="954.025977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:53:39.076393665 +0000 UTC m=+650.530949311" watchObservedRunningTime="2026-07-08 01:53:39.079532927 +0000 UTC m=+650.534088573" Jul 08 01:53:42.326094 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.326055 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8"] Jul 08 01:53:42.328582 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.328556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.331118 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.331095 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 01:53:42.340020 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.339995 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8"] Jul 08 01:53:42.484084 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.484084 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.484295 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.484295 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c746050-af7b-4bdb-9af2-83d7ddb81609-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.484295 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.484295 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.484235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2bz\" (UniqueName: \"kubernetes.io/projected/9c746050-af7b-4bdb-9af2-83d7ddb81609-kube-api-access-9j2bz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585466 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585660 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c746050-af7b-4bdb-9af2-83d7ddb81609-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.585870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2bz\" (UniqueName: \"kubernetes.io/projected/9c746050-af7b-4bdb-9af2-83d7ddb81609-kube-api-access-9j2bz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.586617 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.586881 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.589458 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.591849 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.589718 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c746050-af7b-4bdb-9af2-83d7ddb81609-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.592880 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.592826 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c746050-af7b-4bdb-9af2-83d7ddb81609-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.594868 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.594824 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2bz\" (UniqueName: \"kubernetes.io/projected/9c746050-af7b-4bdb-9af2-83d7ddb81609-kube-api-access-9j2bz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swxz8\" (UID: \"9c746050-af7b-4bdb-9af2-83d7ddb81609\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.641766 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.641718 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:42.776979 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:42.776939 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8"] Jul 08 01:53:42.779439 ip-10-0-136-194 kubenswrapper[2560]: W0708 01:53:42.779402 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c746050_af7b_4bdb_9af2_83d7ddb81609.slice/crio-0c58fb447c79bd6bdf362ab7838143df378e3e9fc4af33af8a6680aef1c9e3fb WatchSource:0}: Error finding container 0c58fb447c79bd6bdf362ab7838143df378e3e9fc4af33af8a6680aef1c9e3fb: Status 404 returned error can't find the container with id 0c58fb447c79bd6bdf362ab7838143df378e3e9fc4af33af8a6680aef1c9e3fb Jul 08 01:53:43.078570 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:43.078531 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" event={"ID":"9c746050-af7b-4bdb-9af2-83d7ddb81609","Type":"ContainerStarted","Data":"a91c3b8fa12c279e76db7975eddb5715d1cfd8a64fd3e6334bfcb4c8e007d462"} Jul 08 01:53:43.078777 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:43.078595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" event={"ID":"9c746050-af7b-4bdb-9af2-83d7ddb81609","Type":"ContainerStarted","Data":"0c58fb447c79bd6bdf362ab7838143df378e3e9fc4af33af8a6680aef1c9e3fb"} Jul 08 01:53:43.097043 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:43.096991 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" podStartSLOduration=0.939555482 podStartE2EDuration="1.096976272s" totalImagesPullingTime="157.42079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 01:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 01:53:43.094981288 +0000 UTC m=+654.549536934" watchObservedRunningTime="2026-07-08 01:53:43.096976272 +0000 UTC m=+654.551531894" Jul 08 01:53:46.929460 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:46.929419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:46.929895 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:46.929477 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:46.942205 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:46.942179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:47.109999 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:47.109975 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kdqrt" Jul 08 01:53:52.642889 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:52.642822 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:52.643299 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:52.642939 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:52.655683 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:52.655646 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 01:53:53.146449 ip-10-0-136-194 kubenswrapper[2560]: I0708 01:53:53.146417 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swxz8" Jul 08 02:00:45.174796 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:45.174709 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 02:00:45.175353 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:45.175008 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" podUID="eff79c7e-5793-4fb6-b4ab-4395a6322cba" containerName="manager" containerID="cri-o://d7409ed12d658510d8b52568d778f97b72e36d9cd1243082d6a9ca624f718c78" gracePeriod=10 Jul 08 02:00:48.529653 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.529623 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 02:00:48.614658 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.614556 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume\") pod \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " Jul 08 02:00:48.614658 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.614616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82hc7\" (UniqueName: \"kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7\") pod \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\" (UID: \"eff79c7e-5793-4fb6-b4ab-4395a6322cba\") " Jul 08 02:00:48.615041 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.615018 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eff79c7e-5793-4fb6-b4ab-4395a6322cba" (UID: "eff79c7e-5793-4fb6-b4ab-4395a6322cba"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 02:00:48.616667 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.616643 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7" (OuterVolumeSpecName: "kube-api-access-82hc7") pod "eff79c7e-5793-4fb6-b4ab-4395a6322cba" (UID: "eff79c7e-5793-4fb6-b4ab-4395a6322cba"). InnerVolumeSpecName "kube-api-access-82hc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:00:48.716032 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.715973 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82hc7\" (UniqueName: \"kubernetes.io/projected/eff79c7e-5793-4fb6-b4ab-4395a6322cba-kube-api-access-82hc7\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:00:48.716032 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.716023 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eff79c7e-5793-4fb6-b4ab-4395a6322cba-extensions-socket-volume\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:00:48.989770 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:48.989684 2560 scope.go:117] "RemoveContainer" containerID="d7409ed12d658510d8b52568d778f97b72e36d9cd1243082d6a9ca624f718c78" Jul 08 02:00:49.196021 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:49.195990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" Jul 08 02:00:49.196021 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:49.195979 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" event={"ID":"eff79c7e-5793-4fb6-b4ab-4395a6322cba","Type":"ContainerDied","Data":"d7409ed12d658510d8b52568d778f97b72e36d9cd1243082d6a9ca624f718c78"} Jul 08 02:00:49.196240 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:49.196050 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv" event={"ID":"eff79c7e-5793-4fb6-b4ab-4395a6322cba","Type":"ContainerDied","Data":"209687c52c1a1791e9cdb5ee168a06056d7f49301c8952834bf78b0bec70d719"} Jul 08 02:00:49.230564 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:49.230528 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 02:00:49.234215 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:49.234187 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6hgv"] Jul 08 02:00:51.029350 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:00:51.029250 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff79c7e-5793-4fb6-b4ab-4395a6322cba" path="/var/lib/kubelet/pods/eff79c7e-5793-4fb6-b4ab-4395a6322cba/volumes" Jul 08 02:01:46.336733 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.336697 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph"] Jul 08 02:01:46.337374 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.337231 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff79c7e-5793-4fb6-b4ab-4395a6322cba" containerName="manager" Jul 08 02:01:46.337374 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.337253 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff79c7e-5793-4fb6-b4ab-4395a6322cba" containerName="manager" Jul 08 02:01:46.337374 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.337338 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff79c7e-5793-4fb6-b4ab-4395a6322cba" containerName="manager" Jul 08 02:01:46.340294 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.340274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.343216 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.343198 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 02:01:46.343330 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.343245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 02:01:46.343330 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.343200 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dhp5g\"" Jul 08 02:01:46.353168 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.353136 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph"] Jul 08 02:01:46.398662 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.398619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2921e97-2aba-41fd-8876-cb58ef622135-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.398872 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.398739 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqwvm\" (UniqueName: \"kubernetes.io/projected/b2921e97-2aba-41fd-8876-cb58ef622135-kube-api-access-kqwvm\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.499756 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.499712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqwvm\" (UniqueName: \"kubernetes.io/projected/b2921e97-2aba-41fd-8876-cb58ef622135-kube-api-access-kqwvm\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.499965 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.499792 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2921e97-2aba-41fd-8876-cb58ef622135-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.500196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.500174 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2921e97-2aba-41fd-8876-cb58ef622135-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.508568 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.508543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqwvm\" (UniqueName: \"kubernetes.io/projected/b2921e97-2aba-41fd-8876-cb58ef622135-kube-api-access-kqwvm\") pod \"kuadrant-operator-controller-manager-86878c769f-tgrph\" (UID: \"b2921e97-2aba-41fd-8876-cb58ef622135\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.650716 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.650617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:46.798118 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:46.798090 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph"] Jul 08 02:01:46.799928 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:01:46.799893 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2921e97_2aba_41fd_8876_cb58ef622135.slice/crio-c13af0f3a88473d0b05f9f4ca6ab9f292188f20aec71c5adf4addbc2b766d743 WatchSource:0}: Error finding container c13af0f3a88473d0b05f9f4ca6ab9f292188f20aec71c5adf4addbc2b766d743: Status 404 returned error can't find the container with id c13af0f3a88473d0b05f9f4ca6ab9f292188f20aec71c5adf4addbc2b766d743 Jul 08 02:01:47.478665 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:47.478622 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" event={"ID":"b2921e97-2aba-41fd-8876-cb58ef622135","Type":"ContainerStarted","Data":"f3b2f85a4cbf70893b1038b5c7eb75a517ac15c16efe34ff4135d39662b77c61"} Jul 08 02:01:47.479123 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:47.478680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" event={"ID":"b2921e97-2aba-41fd-8876-cb58ef622135","Type":"ContainerStarted","Data":"c13af0f3a88473d0b05f9f4ca6ab9f292188f20aec71c5adf4addbc2b766d743"} Jul 08 02:01:47.479123 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:47.478702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:01:47.519767 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:47.519718 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" podStartSLOduration=1.5196986670000001 podStartE2EDuration="1.519698667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:01:47.517254156 +0000 UTC m=+1138.971809802" watchObservedRunningTime="2026-07-08 02:01:47.519698667 +0000 UTC m=+1138.974254324" Jul 08 02:01:58.487462 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:01:58.487425 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tgrph" Jul 08 02:08:32.745168 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.745126 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:32.749094 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.749068 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.751746 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.751719 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-632ff5cb-serving-cert\"" Jul 08 02:08:32.751930 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.751764 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:08:32.752648 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.752628 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:08:32.752757 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.752698 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:08:32.760026 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.759990 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:32.837267 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.837227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55r6\" (UniqueName: \"kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.837453 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.837279 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.837453 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.837315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.937714 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.937670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55r6\" (UniqueName: \"kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.937959 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.937733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.937959 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.937773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.938049 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:32.937962 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-serving-cert: secret "maas-api-e2e-ait-632ff5cb-serving-cert" not found Jul 08 02:08:32.938049 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:32.938035 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls podName:040e883a-2f09-4208-8bfd-f147bad13760 nodeName:}" failed. No retries permitted until 2026-07-08 02:08:33.438018216 +0000 UTC m=+1544.892573843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls") pod "maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" (UID: "040e883a-2f09-4208-8bfd-f147bad13760") : secret "maas-api-e2e-ait-632ff5cb-serving-cert" not found Jul 08 02:08:32.938462 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.938439 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:32.947268 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:32.947233 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55r6\" (UniqueName: \"kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:33.440880 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:33.440822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:33.443520 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:33.443486 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") pod \"maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:33.662502 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:33.662467 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:33.801789 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:33.801771 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:33.804301 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:08:33.804275 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040e883a_2f09_4208_8bfd_f147bad13760.slice/crio-c1f72042f70e24f982005c0f1e77e896ae91cd1d437566f0e8cb34b1b525ba34 WatchSource:0}: Error finding container c1f72042f70e24f982005c0f1e77e896ae91cd1d437566f0e8cb34b1b525ba34: Status 404 returned error can't find the container with id c1f72042f70e24f982005c0f1e77e896ae91cd1d437566f0e8cb34b1b525ba34 Jul 08 02:08:33.805537 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:33.805515 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 02:08:34.528384 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:34.528341 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" event={"ID":"040e883a-2f09-4208-8bfd-f147bad13760","Type":"ContainerStarted","Data":"c1f72042f70e24f982005c0f1e77e896ae91cd1d437566f0e8cb34b1b525ba34"} Jul 08 02:08:36.541337 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:36.541297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" event={"ID":"040e883a-2f09-4208-8bfd-f147bad13760","Type":"ContainerStarted","Data":"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff"} Jul 08 02:08:36.541727 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:36.541392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:36.558181 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:36.558129 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" podStartSLOduration=2.569203408 podStartE2EDuration="4.558111587s" totalImagesPullingTime="1.988908179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:08:36.555787017 +0000 UTC m=+1548.010342664" watchObservedRunningTime="2026-07-08 02:08:36.558111587 +0000 UTC m=+1548.012667232" Jul 08 02:08:40.696289 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.696249 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:40.697182 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.697144 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" podUID="040e883a-2f09-4208-8bfd-f147bad13760" containerName="maas-api" containerID="cri-o://2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff" gracePeriod=30 Jul 08 02:08:40.702914 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.702892 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:40.948591 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.948531 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:40.999150 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.999116 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") pod \"040e883a-2f09-4208-8bfd-f147bad13760\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " Jul 08 02:08:40.999150 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.999168 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n55r6\" (UniqueName: \"kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6\") pod \"040e883a-2f09-4208-8bfd-f147bad13760\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " Jul 08 02:08:40.999430 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.999239 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca\") pod \"040e883a-2f09-4208-8bfd-f147bad13760\" (UID: \"040e883a-2f09-4208-8bfd-f147bad13760\") " Jul 08 02:08:40.999988 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:40.999924 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "040e883a-2f09-4208-8bfd-f147bad13760" (UID: "040e883a-2f09-4208-8bfd-f147bad13760"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:08:41.001948 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.001919 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "040e883a-2f09-4208-8bfd-f147bad13760" (UID: "040e883a-2f09-4208-8bfd-f147bad13760"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:08:41.002072 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.002054 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6" (OuterVolumeSpecName: "kube-api-access-n55r6") pod "040e883a-2f09-4208-8bfd-f147bad13760" (UID: "040e883a-2f09-4208-8bfd-f147bad13760"). InnerVolumeSpecName "kube-api-access-n55r6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:08:41.100162 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.100109 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n55r6\" (UniqueName: \"kubernetes.io/projected/040e883a-2f09-4208-8bfd-f147bad13760-kube-api-access-n55r6\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:41.100162 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.100154 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/040e883a-2f09-4208-8bfd-f147bad13760-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:41.100162 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.100165 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/040e883a-2f09-4208-8bfd-f147bad13760-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:41.569813 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.569778 2560 generic.go:356] "Generic (PLEG): container finished" podID="040e883a-2f09-4208-8bfd-f147bad13760" containerID="2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff" exitCode=0 Jul 08 02:08:41.570008 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.569825 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" Jul 08 02:08:41.570008 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.569875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" event={"ID":"040e883a-2f09-4208-8bfd-f147bad13760","Type":"ContainerDied","Data":"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff"} Jul 08 02:08:41.570008 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.569937 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg" event={"ID":"040e883a-2f09-4208-8bfd-f147bad13760","Type":"ContainerDied","Data":"c1f72042f70e24f982005c0f1e77e896ae91cd1d437566f0e8cb34b1b525ba34"} Jul 08 02:08:41.570008 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.569966 2560 scope.go:117] "RemoveContainer" containerID="2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff" Jul 08 02:08:41.578987 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.578965 2560 scope.go:117] "RemoveContainer" containerID="2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff" Jul 08 02:08:41.579298 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:41.579276 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff\": container with ID starting with 2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff not found: ID does not exist" containerID="2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff" Jul 08 02:08:41.579389 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.579306 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff"} err="failed to get container status \"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff\": rpc error: code = NotFound desc = could not find container \"2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff\": container with ID starting with 2e1333167acee19ecdf5c65addb5546737b4a87ac4f23297f18f7bd21a8dfeff not found: ID does not exist" Jul 08 02:08:41.587153 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.587125 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:41.592425 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:41.592404 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-632ff5cb-7c4f6bfc68-xksjg"] Jul 08 02:08:43.030136 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:43.030097 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="040e883a-2f09-4208-8bfd-f147bad13760" path="/var/lib/kubelet/pods/040e883a-2f09-4208-8bfd-f147bad13760/volumes" Jul 08 02:08:48.051286 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.051252 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:48.051955 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.051932 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="040e883a-2f09-4208-8bfd-f147bad13760" containerName="maas-api" Jul 08 02:08:48.052141 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.052122 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="040e883a-2f09-4208-8bfd-f147bad13760" containerName="maas-api" Jul 08 02:08:48.052267 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.052237 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="040e883a-2f09-4208-8bfd-f147bad13760" containerName="maas-api" Jul 08 02:08:48.056742 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.056721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.059315 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.059291 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:08:48.059529 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.059511 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f51bc25c-serving-cert\"" Jul 08 02:08:48.059744 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.059724 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:08:48.060336 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.060315 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:08:48.065313 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.065282 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:48.160434 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.160400 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58lcf\" (UniqueName: \"kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.160606 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.160448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.160606 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.160531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.261912 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.261873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58lcf\" (UniqueName: \"kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.261912 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.261916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.262189 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.262173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.262274 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:48.262258 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-serving-cert: secret "maas-api-e2e-ait-f51bc25c-serving-cert" not found Jul 08 02:08:48.262323 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:48.262309 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls podName:c9a9a4bb-481a-4621-92cc-034fe30c3bc1 nodeName:}" failed. No retries permitted until 2026-07-08 02:08:48.762293884 +0000 UTC m=+1560.216849508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls") pod "maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" (UID: "c9a9a4bb-481a-4621-92cc-034fe30c3bc1") : secret "maas-api-e2e-ait-f51bc25c-serving-cert" not found Jul 08 02:08:48.262940 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.262917 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.270678 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.270644 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58lcf\" (UniqueName: \"kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.767261 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.767218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.769808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.769780 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") pod \"maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:48.973528 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.973491 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:08:48.982451 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:48.982416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:49.122544 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:49.122420 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:49.125332 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:08:49.125303 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a9a4bb_481a_4621_92cc_034fe30c3bc1.slice/crio-29b9c3fed45a467db0c88c1c78268e8f21a661133926c7bea203cc3424db2cc5 WatchSource:0}: Error finding container 29b9c3fed45a467db0c88c1c78268e8f21a661133926c7bea203cc3424db2cc5: Status 404 returned error can't find the container with id 29b9c3fed45a467db0c88c1c78268e8f21a661133926c7bea203cc3424db2cc5 Jul 08 02:08:49.611302 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:49.611266 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" event={"ID":"c9a9a4bb-481a-4621-92cc-034fe30c3bc1","Type":"ContainerStarted","Data":"29b9c3fed45a467db0c88c1c78268e8f21a661133926c7bea203cc3424db2cc5"} Jul 08 02:08:50.618086 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:50.618049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" event={"ID":"c9a9a4bb-481a-4621-92cc-034fe30c3bc1","Type":"ContainerStarted","Data":"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd"} Jul 08 02:08:50.618502 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:50.618126 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:50.634881 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:50.634812 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" podStartSLOduration=2.108124897 podStartE2EDuration="2.634793397s" totalImagesPullingTime="526.6685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:08:50.634064909 +0000 UTC m=+1562.088620555" watchObservedRunningTime="2026-07-08 02:08:50.634793397 +0000 UTC m=+1562.089349043" Jul 08 02:08:55.973854 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:55.973807 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:55.974318 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:55.974077 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" podUID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" containerName="maas-api" containerID="cri-o://38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd" gracePeriod=30 Jul 08 02:08:55.981225 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:55.981203 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:56.218077 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.218050 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:56.333186 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.333147 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca\") pod \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " Jul 08 02:08:56.333379 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.333262 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") pod \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " Jul 08 02:08:56.333379 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.333294 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58lcf\" (UniqueName: \"kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf\") pod \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\" (UID: \"c9a9a4bb-481a-4621-92cc-034fe30c3bc1\") " Jul 08 02:08:56.333560 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.333535 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9a9a4bb-481a-4621-92cc-034fe30c3bc1" (UID: "c9a9a4bb-481a-4621-92cc-034fe30c3bc1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:08:56.335408 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.335377 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf" (OuterVolumeSpecName: "kube-api-access-58lcf") pod "c9a9a4bb-481a-4621-92cc-034fe30c3bc1" (UID: "c9a9a4bb-481a-4621-92cc-034fe30c3bc1"). InnerVolumeSpecName "kube-api-access-58lcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:08:56.335519 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.335454 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9a9a4bb-481a-4621-92cc-034fe30c3bc1" (UID: "c9a9a4bb-481a-4621-92cc-034fe30c3bc1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:08:56.434145 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.434107 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:56.434145 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.434143 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58lcf\" (UniqueName: \"kubernetes.io/projected/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-kube-api-access-58lcf\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:56.434354 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.434159 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a9a4bb-481a-4621-92cc-034fe30c3bc1-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:08:56.648014 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.647924 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" containerID="38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd" exitCode=0 Jul 08 02:08:56.648014 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.647969 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" Jul 08 02:08:56.648244 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.648014 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" event={"ID":"c9a9a4bb-481a-4621-92cc-034fe30c3bc1","Type":"ContainerDied","Data":"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd"} Jul 08 02:08:56.648244 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.648073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz" event={"ID":"c9a9a4bb-481a-4621-92cc-034fe30c3bc1","Type":"ContainerDied","Data":"29b9c3fed45a467db0c88c1c78268e8f21a661133926c7bea203cc3424db2cc5"} Jul 08 02:08:56.648244 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.648104 2560 scope.go:117] "RemoveContainer" containerID="38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd" Jul 08 02:08:56.657899 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.657882 2560 scope.go:117] "RemoveContainer" containerID="38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd" Jul 08 02:08:56.658197 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:08:56.658177 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd\": container with ID starting with 38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd not found: ID does not exist" containerID="38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd" Jul 08 02:08:56.658252 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.658205 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd"} err="failed to get container status \"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd\": rpc error: code = NotFound desc = could not find container \"38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd\": container with ID starting with 38390bb1ae7a940e6fe148cc24769a50ecb6243ccff8fe01988e2d2c7797ebcd not found: ID does not exist" Jul 08 02:08:56.670273 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.670000 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:56.672448 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:56.672425 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f51bc25c-7788788bbd-fz2nz"] Jul 08 02:08:57.029390 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:08:57.029295 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" path="/var/lib/kubelet/pods/c9a9a4bb-481a-4621-92cc-034fe30c3bc1/volumes" Jul 08 02:09:04.128684 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.128649 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:04.129071 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.129027 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" containerName="maas-api" Jul 08 02:09:04.129071 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.129041 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" containerName="maas-api" Jul 08 02:09:04.129145 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.129120 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a9a4bb-481a-4621-92cc-034fe30c3bc1" containerName="maas-api" Jul 08 02:09:04.133154 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.133135 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.135933 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.135909 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:09:04.136650 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.136629 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:09:04.136770 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.136632 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2df057ea-serving-cert\"" Jul 08 02:09:04.136770 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.136634 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:09:04.142309 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.142283 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:04.200225 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.200186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.200409 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.200242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.200409 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.200272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt8bb\" (UniqueName: \"kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.301681 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.301642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.301681 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.301689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt8bb\" (UniqueName: \"kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.301923 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.301739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.301923 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:09:04.301792 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-serving-cert: secret "maas-api-e2e-derive-2df057ea-serving-cert" not found Jul 08 02:09:04.301923 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:09:04.301897 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls podName:b151c2fe-e320-4e94-8fce-48b688604b97 nodeName:}" failed. No retries permitted until 2026-07-08 02:09:04.801875766 +0000 UTC m=+1576.256431398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls") pod "maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" (UID: "b151c2fe-e320-4e94-8fce-48b688604b97") : secret "maas-api-e2e-derive-2df057ea-serving-cert" not found Jul 08 02:09:04.302473 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.302451 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.310068 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.310040 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt8bb\" (UniqueName: \"kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.809632 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.809589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:04.809827 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:04.809726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") pod \"maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:05.045091 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.045050 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:05.201826 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.201797 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:05.204280 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:09:05.204251 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb151c2fe_e320_4e94_8fce_48b688604b97.slice/crio-09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66 WatchSource:0}: Error finding container 09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66: Status 404 returned error can't find the container with id 09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66 Jul 08 02:09:05.700360 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.700325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" event={"ID":"b151c2fe-e320-4e94-8fce-48b688604b97","Type":"ContainerStarted","Data":"e50c81f6dda68b525246830e610c2a58c986035cd5f1c1f49d7c5c9038c81c02"} Jul 08 02:09:05.700512 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.700378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" event={"ID":"b151c2fe-e320-4e94-8fce-48b688604b97","Type":"ContainerStarted","Data":"09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66"} Jul 08 02:09:05.700512 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.700500 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:05.718320 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:05.718270 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" podStartSLOduration=1.319030506 podStartE2EDuration="1.718254102s" totalImagesPullingTime="399.223596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:09:05.71540955 +0000 UTC m=+1577.169965195" watchObservedRunningTime="2026-07-08 02:09:05.718254102 +0000 UTC m=+1577.172809748" Jul 08 02:09:11.494820 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.494781 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:11.495377 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.495059 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" podUID="b151c2fe-e320-4e94-8fce-48b688604b97" containerName="maas-api" containerID="cri-o://e50c81f6dda68b525246830e610c2a58c986035cd5f1c1f49d7c5c9038c81c02" gracePeriod=30 Jul 08 02:09:11.501819 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.501794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:11.734712 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.734676 2560 generic.go:356] "Generic (PLEG): container finished" podID="b151c2fe-e320-4e94-8fce-48b688604b97" containerID="e50c81f6dda68b525246830e610c2a58c986035cd5f1c1f49d7c5c9038c81c02" exitCode=0 Jul 08 02:09:11.734888 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.734747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" event={"ID":"b151c2fe-e320-4e94-8fce-48b688604b97","Type":"ContainerDied","Data":"e50c81f6dda68b525246830e610c2a58c986035cd5f1c1f49d7c5c9038c81c02"} Jul 08 02:09:11.734888 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.734816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" event={"ID":"b151c2fe-e320-4e94-8fce-48b688604b97","Type":"ContainerDied","Data":"09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66"} Jul 08 02:09:11.734888 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.734857 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09885b8b419e8aceb770cc096d0adcda71e0fb0d128996e81dda4dd9f66dcc66" Jul 08 02:09:11.745771 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.745714 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:11.871407 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.871373 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") pod \"b151c2fe-e320-4e94-8fce-48b688604b97\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " Jul 08 02:09:11.871706 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.871448 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca\") pod \"b151c2fe-e320-4e94-8fce-48b688604b97\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " Jul 08 02:09:11.871706 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.871594 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt8bb\" (UniqueName: \"kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb\") pod \"b151c2fe-e320-4e94-8fce-48b688604b97\" (UID: \"b151c2fe-e320-4e94-8fce-48b688604b97\") " Jul 08 02:09:11.871826 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.871806 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b151c2fe-e320-4e94-8fce-48b688604b97" (UID: "b151c2fe-e320-4e94-8fce-48b688604b97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:09:11.873620 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.873589 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b151c2fe-e320-4e94-8fce-48b688604b97" (UID: "b151c2fe-e320-4e94-8fce-48b688604b97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:09:11.873734 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.873648 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb" (OuterVolumeSpecName: "kube-api-access-lt8bb") pod "b151c2fe-e320-4e94-8fce-48b688604b97" (UID: "b151c2fe-e320-4e94-8fce-48b688604b97"). InnerVolumeSpecName "kube-api-access-lt8bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:09:11.972295 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.972255 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt8bb\" (UniqueName: \"kubernetes.io/projected/b151c2fe-e320-4e94-8fce-48b688604b97-kube-api-access-lt8bb\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:09:11.972295 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.972289 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b151c2fe-e320-4e94-8fce-48b688604b97-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:09:11.972295 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:11.972298 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b151c2fe-e320-4e94-8fce-48b688604b97-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:09:12.739337 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:12.739304 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf" Jul 08 02:09:12.759535 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:12.759501 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:12.763666 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:12.763638 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2df057ea-78dc9dd5f4-8sjpf"] Jul 08 02:09:13.029516 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:09:13.029417 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b151c2fe-e320-4e94-8fce-48b688604b97" path="/var/lib/kubelet/pods/b151c2fe-e320-4e94-8fce-48b688604b97/volumes" Jul 08 02:12:44.580144 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.580065 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:44.582404 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.580416 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b151c2fe-e320-4e94-8fce-48b688604b97" containerName="maas-api" Jul 08 02:12:44.582404 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.580428 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b151c2fe-e320-4e94-8fce-48b688604b97" containerName="maas-api" Jul 08 02:12:44.582404 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.580487 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b151c2fe-e320-4e94-8fce-48b688604b97" containerName="maas-api" Jul 08 02:12:44.583403 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.583383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.588053 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.588024 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:12:44.588216 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.588192 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b7e0fb27-serving-cert\"" Jul 08 02:12:44.588797 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.588780 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:12:44.589071 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.589057 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:12:44.606474 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.606440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:44.646894 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.646825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.646894 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.646892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m6vq\" (UniqueName: \"kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.647178 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.646979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.747860 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.747796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.748062 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.747956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.748062 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.747985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m6vq\" (UniqueName: \"kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.748176 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:12:44.748147 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-serving-cert: secret "maas-api-e2e-mt-b7e0fb27-serving-cert" not found Jul 08 02:12:44.748245 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:12:44.748230 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls podName:d35ef55d-b035-47e0-aedd-be5cf8776e51 nodeName:}" failed. No retries permitted until 2026-07-08 02:12:45.248206207 +0000 UTC m=+1796.702761835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls") pod "maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" (UID: "d35ef55d-b035-47e0-aedd-be5cf8776e51") : secret "maas-api-e2e-mt-b7e0fb27-serving-cert" not found Jul 08 02:12:44.748586 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.748565 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:44.763773 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:44.763745 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m6vq\" (UniqueName: \"kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:45.253063 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:45.253018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:45.255472 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:45.255444 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") pod \"maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:45.494510 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:45.494468 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:45.629657 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:45.629620 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:45.632803 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:12:45.632775 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35ef55d_b035_47e0_aedd_be5cf8776e51.slice/crio-6beedf2e72ce9d24dc894e2713d5befaa475a487dc57bbe56a5c9af81d82f000 WatchSource:0}: Error finding container 6beedf2e72ce9d24dc894e2713d5befaa475a487dc57bbe56a5c9af81d82f000: Status 404 returned error can't find the container with id 6beedf2e72ce9d24dc894e2713d5befaa475a487dc57bbe56a5c9af81d82f000 Jul 08 02:12:45.834426 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:45.834387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" event={"ID":"d35ef55d-b035-47e0-aedd-be5cf8776e51","Type":"ContainerStarted","Data":"6beedf2e72ce9d24dc894e2713d5befaa475a487dc57bbe56a5c9af81d82f000"} Jul 08 02:12:46.841039 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:46.840997 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" event={"ID":"d35ef55d-b035-47e0-aedd-be5cf8776e51","Type":"ContainerStarted","Data":"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052"} Jul 08 02:12:46.841446 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:46.841069 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:46.859156 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:46.859105 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" podStartSLOduration=2.408649988 podStartE2EDuration="2.859085557s" totalImagesPullingTime="450.435569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:12:46.858299078 +0000 UTC m=+1798.312854723" watchObservedRunningTime="2026-07-08 02:12:46.859085557 +0000 UTC m=+1798.313641205" Jul 08 02:12:52.852534 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:52.852502 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:56.535253 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.535211 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:56.535741 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.535437 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" podUID="d35ef55d-b035-47e0-aedd-be5cf8776e51" containerName="maas-api" containerID="cri-o://bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052" gracePeriod=30 Jul 08 02:12:56.786803 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.786731 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:56.894217 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.894178 2560 generic.go:356] "Generic (PLEG): container finished" podID="d35ef55d-b035-47e0-aedd-be5cf8776e51" containerID="bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052" exitCode=0 Jul 08 02:12:56.894217 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.894223 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" Jul 08 02:12:56.894426 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.894251 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" event={"ID":"d35ef55d-b035-47e0-aedd-be5cf8776e51","Type":"ContainerDied","Data":"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052"} Jul 08 02:12:56.894426 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.894300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp" event={"ID":"d35ef55d-b035-47e0-aedd-be5cf8776e51","Type":"ContainerDied","Data":"6beedf2e72ce9d24dc894e2713d5befaa475a487dc57bbe56a5c9af81d82f000"} Jul 08 02:12:56.894426 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.894325 2560 scope.go:117] "RemoveContainer" containerID="bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052" Jul 08 02:12:56.903491 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.903466 2560 scope.go:117] "RemoveContainer" containerID="bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052" Jul 08 02:12:56.903788 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:12:56.903770 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052\": container with ID starting with bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052 not found: ID does not exist" containerID="bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052" Jul 08 02:12:56.903852 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.903798 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052"} err="failed to get container status \"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052\": rpc error: code = NotFound desc = could not find container \"bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052\": container with ID starting with bda83b704e0fed15550904d83b382771bc1e7312e6bad1dc96bab7d48b08f052 not found: ID does not exist" Jul 08 02:12:56.963268 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.963230 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m6vq\" (UniqueName: \"kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq\") pod \"d35ef55d-b035-47e0-aedd-be5cf8776e51\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " Jul 08 02:12:56.963465 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.963326 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") pod \"d35ef55d-b035-47e0-aedd-be5cf8776e51\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " Jul 08 02:12:56.963465 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.963364 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca\") pod \"d35ef55d-b035-47e0-aedd-be5cf8776e51\" (UID: \"d35ef55d-b035-47e0-aedd-be5cf8776e51\") " Jul 08 02:12:56.963792 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.963756 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d35ef55d-b035-47e0-aedd-be5cf8776e51" (UID: "d35ef55d-b035-47e0-aedd-be5cf8776e51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:12:56.965500 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.965468 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d35ef55d-b035-47e0-aedd-be5cf8776e51" (UID: "d35ef55d-b035-47e0-aedd-be5cf8776e51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:12:56.965607 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:56.965557 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq" (OuterVolumeSpecName: "kube-api-access-6m6vq") pod "d35ef55d-b035-47e0-aedd-be5cf8776e51" (UID: "d35ef55d-b035-47e0-aedd-be5cf8776e51"). InnerVolumeSpecName "kube-api-access-6m6vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:12:57.064219 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:57.064176 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35ef55d-b035-47e0-aedd-be5cf8776e51-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:12:57.064219 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:57.064212 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35ef55d-b035-47e0-aedd-be5cf8776e51-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:12:57.064219 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:57.064226 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m6vq\" (UniqueName: \"kubernetes.io/projected/d35ef55d-b035-47e0-aedd-be5cf8776e51-kube-api-access-6m6vq\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:12:57.211080 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:57.211044 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:57.215146 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:57.215117 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7e0fb27-65b497ffc7-6spdp"] Jul 08 02:12:59.029489 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:12:59.029457 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35ef55d-b035-47e0-aedd-be5cf8776e51" path="/var/lib/kubelet/pods/d35ef55d-b035-47e0-aedd-be5cf8776e51/volumes" Jul 08 02:13:25.739670 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.739634 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:13:25.740168 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.739991 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35ef55d-b035-47e0-aedd-be5cf8776e51" containerName="maas-api" Jul 08 02:13:25.740168 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.740004 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35ef55d-b035-47e0-aedd-be5cf8776e51" containerName="maas-api" Jul 08 02:13:25.740168 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.740065 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d35ef55d-b035-47e0-aedd-be5cf8776e51" containerName="maas-api" Jul 08 02:13:25.743276 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.743258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.745758 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.745731 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b64edcbf-serving-cert\"" Jul 08 02:13:25.745913 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.745861 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:13:25.746131 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.746108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:13:25.746662 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.746643 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:13:25.752880 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.752856 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:13:25.825404 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.825369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj8l4\" (UniqueName: \"kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.825588 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.825423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.825588 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.825536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.926684 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.926642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.926904 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.926740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.926904 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.926772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj8l4\" (UniqueName: \"kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.926904 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:13:25.926814 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-serving-cert: secret "maas-api-e2e-mt-b64edcbf-serving-cert" not found Jul 08 02:13:25.927006 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:13:25.926914 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls podName:7dcd2d4d-92ef-465c-ac03-9b32a615fdd4 nodeName:}" failed. No retries permitted until 2026-07-08 02:13:26.42689234 +0000 UTC m=+1837.881447974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls") pod "maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" (UID: "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4") : secret "maas-api-e2e-mt-b64edcbf-serving-cert" not found Jul 08 02:13:25.927409 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.927386 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:25.936894 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:25.936872 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj8l4\" (UniqueName: \"kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:26.431512 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:26.431458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:26.434040 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:26.434017 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") pod \"maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:26.655703 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:26.655667 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:26.800190 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:26.800159 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:13:26.802608 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:13:26.802581 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dcd2d4d_92ef_465c_ac03_9b32a615fdd4.slice/crio-f20f8b36564c6e5e912dda8751341aad4e6614d530dc555797d3b91489049303 WatchSource:0}: Error finding container f20f8b36564c6e5e912dda8751341aad4e6614d530dc555797d3b91489049303: Status 404 returned error can't find the container with id f20f8b36564c6e5e912dda8751341aad4e6614d530dc555797d3b91489049303 Jul 08 02:13:27.057606 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:27.057569 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" event={"ID":"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4","Type":"ContainerStarted","Data":"f20f8b36564c6e5e912dda8751341aad4e6614d530dc555797d3b91489049303"} Jul 08 02:13:28.064545 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:28.064505 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" event={"ID":"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4","Type":"ContainerStarted","Data":"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8"} Jul 08 02:13:28.064972 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:28.064567 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:13:28.088308 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:28.083718 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" podStartSLOduration=2.688338199 podStartE2EDuration="3.083698301s" totalImagesPullingTime="395.360102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:13:28.080004356 +0000 UTC m=+1839.534560002" watchObservedRunningTime="2026-07-08 02:13:28.083698301 +0000 UTC m=+1839.538253948" Jul 08 02:13:34.077036 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:13:34.077009 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:14:08.164047 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.164007 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:14:08.164645 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.164277 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" podUID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" containerName="maas-api" containerID="cri-o://95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8" gracePeriod=30 Jul 08 02:14:08.413608 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.413580 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:14:08.511649 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.511554 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj8l4\" (UniqueName: \"kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4\") pod \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " Jul 08 02:14:08.511649 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.511628 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca\") pod \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " Jul 08 02:14:08.511649 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.511650 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") pod \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\" (UID: \"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4\") " Jul 08 02:14:08.512050 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.512024 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" (UID: "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:14:08.513810 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.513780 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" (UID: "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:14:08.513935 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.513818 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4" (OuterVolumeSpecName: "kube-api-access-cj8l4") pod "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" (UID: "7dcd2d4d-92ef-465c-ac03-9b32a615fdd4"). InnerVolumeSpecName "kube-api-access-cj8l4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:14:08.613127 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.613083 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:14:08.613127 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.613121 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:14:08.613127 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:08.613130 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj8l4\" (UniqueName: \"kubernetes.io/projected/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4-kube-api-access-cj8l4\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:14:09.279487 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.279450 2560 generic.go:356] "Generic (PLEG): container finished" podID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" containerID="95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8" exitCode=0 Jul 08 02:14:09.279997 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.279497 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" Jul 08 02:14:09.279997 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.279535 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" event={"ID":"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4","Type":"ContainerDied","Data":"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8"} Jul 08 02:14:09.279997 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.279589 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq" event={"ID":"7dcd2d4d-92ef-465c-ac03-9b32a615fdd4","Type":"ContainerDied","Data":"f20f8b36564c6e5e912dda8751341aad4e6614d530dc555797d3b91489049303"} Jul 08 02:14:09.279997 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.279615 2560 scope.go:117] "RemoveContainer" containerID="95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8" Jul 08 02:14:09.290485 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.290430 2560 scope.go:117] "RemoveContainer" containerID="95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8" Jul 08 02:14:09.290877 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:14:09.290824 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8\": container with ID starting with 95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8 not found: ID does not exist" containerID="95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8" Jul 08 02:14:09.290966 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.290888 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8"} err="failed to get container status \"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8\": rpc error: code = NotFound desc = could not find container \"95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8\": container with ID starting with 95bd014c111e76fee621cb5603a51dc9d0c0dbeeb537b2b634ce409789c0dab8 not found: ID does not exist" Jul 08 02:14:09.294877 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.294852 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:14:09.300196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:09.300176 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b64edcbf-58497fb65b-lhvlq"] Jul 08 02:14:11.029194 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:14:11.029094 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" path="/var/lib/kubelet/pods/7dcd2d4d-92ef-465c-ac03-9b32a615fdd4/volumes" Jul 08 02:15:00.138762 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.138725 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:15:00.139251 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.139065 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" containerName="maas-api" Jul 08 02:15:00.139251 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.139077 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" containerName="maas-api" Jul 08 02:15:00.139251 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.139140 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dcd2d4d-92ef-465c-ac03-9b32a615fdd4" containerName="maas-api" Jul 08 02:15:00.142021 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.142003 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:15:00.144428 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.144401 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:15:00.144567 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.144430 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:15:00.144567 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.144431 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:15:00.150043 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.149969 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:15:00.268327 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.268282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghlp\" (UniqueName: \"kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp\") pod \"maas-api-key-cleanup-29724615-jtrjb\" (UID: \"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:15:00.369066 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.369025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghlp\" (UniqueName: \"kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp\") pod \"maas-api-key-cleanup-29724615-jtrjb\" (UID: \"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:15:00.377497 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.377466 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghlp\" (UniqueName: \"kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp\") pod \"maas-api-key-cleanup-29724615-jtrjb\" (UID: \"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:15:00.454222 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.454125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:15:00.591352 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.591319 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:15:00.594811 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:15:00.594784 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a6ef92_6ad7_4fa6_a2f4_54cc18c229a8.slice/crio-8f503c86f39f5f27ec93a90c853d1de14adcc72d0bc7d7199a3696c945790da6 WatchSource:0}: Error finding container 8f503c86f39f5f27ec93a90c853d1de14adcc72d0bc7d7199a3696c945790da6: Status 404 returned error can't find the container with id 8f503c86f39f5f27ec93a90c853d1de14adcc72d0bc7d7199a3696c945790da6 Jul 08 02:15:00.596854 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:00.596826 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 02:15:01.558540 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:01.558498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" event={"ID":"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8","Type":"ContainerStarted","Data":"8f503c86f39f5f27ec93a90c853d1de14adcc72d0bc7d7199a3696c945790da6"} Jul 08 02:15:03.572864 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:03.572806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" event={"ID":"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8","Type":"ContainerStarted","Data":"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55"} Jul 08 02:15:03.590523 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:03.590465 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" podStartSLOduration=1.04635803 podStartE2EDuration="3.590445887s" totalImagesPullingTime="2.544087857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:15:03.58851614 +0000 UTC m=+1935.043071786" watchObservedRunningTime="2026-07-08 02:15:03.590445887 +0000 UTC m=+1935.045001532" Jul 08 02:15:49.145203 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:15:49.145120 2560 scope.go:117] "RemoveContainer" containerID="e50c81f6dda68b525246830e610c2a58c986035cd5f1c1f49d7c5c9038c81c02" Jul 08 02:16:00.011858 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:00.011795 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:16:00.012397 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:00.012071 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" podUID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" containerName="cleanup" containerID="cri-o://f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55" gracePeriod=30 Jul 08 02:16:30.178376 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:30.178344 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:16:30.277247 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:30.277158 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mghlp\" (UniqueName: \"kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp\") pod \"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8\" (UID: \"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8\") " Jul 08 02:16:30.279274 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:30.279247 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp" (OuterVolumeSpecName: "kube-api-access-mghlp") pod "65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" (UID: "65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8"). InnerVolumeSpecName "kube-api-access-mghlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:16:30.378806 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:30.378762 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mghlp\" (UniqueName: \"kubernetes.io/projected/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8-kube-api-access-mghlp\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:16:31.027603 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.027568 2560 generic.go:356] "Generic (PLEG): container finished" podID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" containerID="f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55" exitCode=137 Jul 08 02:16:31.027813 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.027680 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" Jul 08 02:16:31.029870 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.029822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" event={"ID":"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8","Type":"ContainerDied","Data":"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55"} Jul 08 02:16:31.030029 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.029890 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb" event={"ID":"65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8","Type":"ContainerDied","Data":"8f503c86f39f5f27ec93a90c853d1de14adcc72d0bc7d7199a3696c945790da6"} Jul 08 02:16:31.030029 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.029912 2560 scope.go:117] "RemoveContainer" containerID="f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55" Jul 08 02:16:31.039742 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.039722 2560 scope.go:117] "RemoveContainer" containerID="f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55" Jul 08 02:16:31.040080 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:16:31.040060 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55\": container with ID starting with f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55 not found: ID does not exist" containerID="f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55" Jul 08 02:16:31.040160 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.040093 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55"} err="failed to get container status \"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55\": rpc error: code = NotFound desc = could not find container \"f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55\": container with ID starting with f0d79e713e7b258f3e19701ef6ae51a3c5b81e9fd8f953682392b0d8533bfc55 not found: ID does not exist" Jul 08 02:16:31.051741 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.051704 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:16:31.054385 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:31.054360 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29724615-jtrjb"] Jul 08 02:16:33.029747 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:16:33.029709 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" path="/var/lib/kubelet/pods/65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8/volumes" Jul 08 02:17:10.779959 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.779878 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:17:10.780458 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.780226 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" containerName="cleanup" Jul 08 02:17:10.780458 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.780237 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" containerName="cleanup" Jul 08 02:17:10.780458 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.780292 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a6ef92-6ad7-4fa6-a2f4-54cc18c229a8" containerName="cleanup" Jul 08 02:17:10.782301 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.782282 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.785209 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.785180 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7c26aa-serving-cert\"" Jul 08 02:17:10.785405 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.785187 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:17:10.785856 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.785811 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:17:10.785969 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.785876 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:17:10.795987 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.795962 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:17:10.835610 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.835567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.835795 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.835647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.835795 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.835665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkgqb\" (UniqueName: \"kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.936288 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.936251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.936288 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.936288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkgqb\" (UniqueName: \"kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.936522 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.936355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.936522 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:17:10.936407 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-serving-cert: secret "maas-api-e2e-inf-a-7c26aa-serving-cert" not found Jul 08 02:17:10.936522 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:17:10.936475 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls podName:0ea73384-51d5-43b0-9413-4be04995aa31 nodeName:}" failed. No retries permitted until 2026-07-08 02:17:11.436457906 +0000 UTC m=+2062.891013530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls") pod "maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" (UID: "0ea73384-51d5-43b0-9413-4be04995aa31") : secret "maas-api-e2e-inf-a-7c26aa-serving-cert" not found Jul 08 02:17:10.936980 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.936961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:10.946798 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:10.946763 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkgqb\" (UniqueName: \"kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:11.441411 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:11.441375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:11.443809 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:11.443785 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:11.693066 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:11.692972 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:11.826808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:11.826781 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:17:11.829297 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:17:11.829270 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea73384_51d5_43b0_9413_4be04995aa31.slice/crio-f1750d03a7080db34317e279fdd401a53802525a12a0d68db07d8a5d47ebb040 WatchSource:0}: Error finding container f1750d03a7080db34317e279fdd401a53802525a12a0d68db07d8a5d47ebb040: Status 404 returned error can't find the container with id f1750d03a7080db34317e279fdd401a53802525a12a0d68db07d8a5d47ebb040 Jul 08 02:17:12.236672 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:12.236639 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" event={"ID":"0ea73384-51d5-43b0-9413-4be04995aa31","Type":"ContainerStarted","Data":"f1750d03a7080db34317e279fdd401a53802525a12a0d68db07d8a5d47ebb040"} Jul 08 02:17:13.244356 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:13.244315 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" event={"ID":"0ea73384-51d5-43b0-9413-4be04995aa31","Type":"ContainerStarted","Data":"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c"} Jul 08 02:17:13.244763 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:13.244456 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:13.260982 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:13.260932 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" podStartSLOduration=2.784444426 podStartE2EDuration="3.260912906s" totalImagesPullingTime="476.46848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:17:13.26024531 +0000 UTC m=+2064.714800956" watchObservedRunningTime="2026-07-08 02:17:13.260912906 +0000 UTC m=+2064.715468552" Jul 08 02:17:18.967105 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:18.967061 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:17:18.969979 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:18.969946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:18.972251 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:18.972227 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-219487-serving-cert\"" Jul 08 02:17:18.980409 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:18.980378 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:17:19.114085 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.114042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.114276 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.114119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgjgh\" (UniqueName: \"kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.114276 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.114143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.215065 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.215018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgjgh\" (UniqueName: \"kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.215271 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.215080 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.215271 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.215137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.215852 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.215816 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.217571 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.217512 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.223796 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.223769 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgjgh\" (UniqueName: \"kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh\") pod \"maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.256920 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.256870 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:17:19.282290 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.282259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:19.428920 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:19.428865 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:17:19.431424 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:17:19.431396 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c7e2c6_6e53_4484_8a90_4a5a215118fd.slice/crio-6a2c1dd43dab569b874f5911d8747da5e77d0c317f069eef7e8543f94d756823 WatchSource:0}: Error finding container 6a2c1dd43dab569b874f5911d8747da5e77d0c317f069eef7e8543f94d756823: Status 404 returned error can't find the container with id 6a2c1dd43dab569b874f5911d8747da5e77d0c317f069eef7e8543f94d756823 Jul 08 02:17:20.282730 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:20.282693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" event={"ID":"e5c7e2c6-6e53-4484-8a90-4a5a215118fd","Type":"ContainerStarted","Data":"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1"} Jul 08 02:17:20.283255 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:20.282743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" event={"ID":"e5c7e2c6-6e53-4484-8a90-4a5a215118fd","Type":"ContainerStarted","Data":"6a2c1dd43dab569b874f5911d8747da5e77d0c317f069eef7e8543f94d756823"} Jul 08 02:17:20.283255 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:20.282826 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:17:20.300480 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:20.300422 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" podStartSLOduration=1.951770572 podStartE2EDuration="2.300407232s" totalImagesPullingTime="348.63666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:17:20.298484311 +0000 UTC m=+2071.753039957" watchObservedRunningTime="2026-07-08 02:17:20.300407232 +0000 UTC m=+2071.754962877" Jul 08 02:17:26.292929 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:17:26.292897 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:18:50.337600 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.337501 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:18:50.340149 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.340129 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.342684 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.342662 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-f124b1-serving-cert\"" Jul 08 02:18:50.354784 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.354755 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:18:50.420621 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.420589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2ljs\" (UniqueName: \"kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.420802 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.420634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.420802 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.420664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.521358 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.521314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.521544 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.521424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2ljs\" (UniqueName: \"kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.521544 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.521455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.521659 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:18:50.521570 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-serving-cert: secret "maas-api-e2e-reuse-f124b1-serving-cert" not found Jul 08 02:18:50.521659 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:18:50.521646 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls podName:a06cb567-1932-495c-89fc-56abd6ac58c9 nodeName:}" failed. No retries permitted until 2026-07-08 02:18:51.021624507 +0000 UTC m=+2162.476180140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls") pod "maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" (UID: "a06cb567-1932-495c-89fc-56abd6ac58c9") : secret "maas-api-e2e-reuse-f124b1-serving-cert" not found Jul 08 02:18:50.522021 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.521999 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:50.532451 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:50.532417 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2ljs\" (UniqueName: \"kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:51.026395 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:51.026356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:51.029018 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:51.028984 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") pod \"maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:51.250552 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:51.250509 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:51.389286 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:51.389259 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:18:51.391414 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:18:51.391388 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06cb567_1932_495c_89fc_56abd6ac58c9.slice/crio-b9d79e441f594e87ea80b699bb2df6af8cf54b33ba9169e2495439e8a7c609a0 WatchSource:0}: Error finding container b9d79e441f594e87ea80b699bb2df6af8cf54b33ba9169e2495439e8a7c609a0: Status 404 returned error can't find the container with id b9d79e441f594e87ea80b699bb2df6af8cf54b33ba9169e2495439e8a7c609a0 Jul 08 02:18:51.745826 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:51.745789 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" event={"ID":"a06cb567-1932-495c-89fc-56abd6ac58c9","Type":"ContainerStarted","Data":"b9d79e441f594e87ea80b699bb2df6af8cf54b33ba9169e2495439e8a7c609a0"} Jul 08 02:18:52.752480 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:52.752437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" event={"ID":"a06cb567-1932-495c-89fc-56abd6ac58c9","Type":"ContainerStarted","Data":"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01"} Jul 08 02:18:52.752941 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:52.752508 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:18:52.773125 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:52.773075 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" podStartSLOduration=2.394381334 podStartE2EDuration="2.773057688s" totalImagesPullingTime="378.676354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:18:52.771232792 +0000 UTC m=+2164.225788449" watchObservedRunningTime="2026-07-08 02:18:52.773057688 +0000 UTC m=+2164.227613312" Jul 08 02:18:58.764731 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:18:58.764698 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:19:17.248302 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.248263 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:19:17.248812 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.248565 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" podUID="a06cb567-1932-495c-89fc-56abd6ac58c9" containerName="maas-api" containerID="cri-o://8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01" gracePeriod=30 Jul 08 02:19:17.503955 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.503883 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:19:17.572746 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.572708 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca\") pod \"a06cb567-1932-495c-89fc-56abd6ac58c9\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " Jul 08 02:19:17.572746 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.572754 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2ljs\" (UniqueName: \"kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs\") pod \"a06cb567-1932-495c-89fc-56abd6ac58c9\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " Jul 08 02:19:17.572999 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.572788 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") pod \"a06cb567-1932-495c-89fc-56abd6ac58c9\" (UID: \"a06cb567-1932-495c-89fc-56abd6ac58c9\") " Jul 08 02:19:17.573135 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.573110 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a06cb567-1932-495c-89fc-56abd6ac58c9" (UID: "a06cb567-1932-495c-89fc-56abd6ac58c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:19:17.574975 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.574939 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs" (OuterVolumeSpecName: "kube-api-access-k2ljs") pod "a06cb567-1932-495c-89fc-56abd6ac58c9" (UID: "a06cb567-1932-495c-89fc-56abd6ac58c9"). InnerVolumeSpecName "kube-api-access-k2ljs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:19:17.574975 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.574944 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a06cb567-1932-495c-89fc-56abd6ac58c9" (UID: "a06cb567-1932-495c-89fc-56abd6ac58c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:19:17.674548 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.674507 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06cb567-1932-495c-89fc-56abd6ac58c9-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:19:17.674548 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.674546 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2ljs\" (UniqueName: \"kubernetes.io/projected/a06cb567-1932-495c-89fc-56abd6ac58c9-kube-api-access-k2ljs\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:19:17.674787 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.674561 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06cb567-1932-495c-89fc-56abd6ac58c9-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:19:17.891883 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.891811 2560 generic.go:356] "Generic (PLEG): container finished" podID="a06cb567-1932-495c-89fc-56abd6ac58c9" containerID="8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01" exitCode=0 Jul 08 02:19:17.892095 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.891890 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" Jul 08 02:19:17.892095 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.891906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" event={"ID":"a06cb567-1932-495c-89fc-56abd6ac58c9","Type":"ContainerDied","Data":"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01"} Jul 08 02:19:17.892095 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.891974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25" event={"ID":"a06cb567-1932-495c-89fc-56abd6ac58c9","Type":"ContainerDied","Data":"b9d79e441f594e87ea80b699bb2df6af8cf54b33ba9169e2495439e8a7c609a0"} Jul 08 02:19:17.892095 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.892005 2560 scope.go:117] "RemoveContainer" containerID="8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01" Jul 08 02:19:17.902682 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.902659 2560 scope.go:117] "RemoveContainer" containerID="8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01" Jul 08 02:19:17.902994 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:19:17.902975 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01\": container with ID starting with 8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01 not found: ID does not exist" containerID="8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01" Jul 08 02:19:17.903073 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.903002 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01"} err="failed to get container status \"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01\": rpc error: code = NotFound desc = could not find container \"8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01\": container with ID starting with 8cd4383051b97153d7a636d5ca9ccd4f0b4b558c80e1c301911ad3760fd36d01 not found: ID does not exist" Jul 08 02:19:17.920249 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.920215 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:19:17.927998 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:17.927965 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-f124b1-5c4b6686d9-pzq25"] Jul 08 02:19:19.030849 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:19:19.030803 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a06cb567-1932-495c-89fc-56abd6ac58c9" path="/var/lib/kubelet/pods/a06cb567-1932-495c-89fc-56abd6ac58c9/volumes" Jul 08 02:20:31.865266 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:31.865058 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:20:31.866057 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:31.866025 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" podUID="0ea73384-51d5-43b0-9413-4be04995aa31" containerName="maas-api" containerID="cri-o://d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c" gracePeriod=30 Jul 08 02:20:32.114134 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.114107 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:20:32.151149 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.151067 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") pod \"0ea73384-51d5-43b0-9413-4be04995aa31\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " Jul 08 02:20:32.151149 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.151125 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca\") pod \"0ea73384-51d5-43b0-9413-4be04995aa31\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " Jul 08 02:20:32.151351 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.151153 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkgqb\" (UniqueName: \"kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb\") pod \"0ea73384-51d5-43b0-9413-4be04995aa31\" (UID: \"0ea73384-51d5-43b0-9413-4be04995aa31\") " Jul 08 02:20:32.151498 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.151474 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ea73384-51d5-43b0-9413-4be04995aa31" (UID: "0ea73384-51d5-43b0-9413-4be04995aa31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:20:32.153302 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.153266 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb" (OuterVolumeSpecName: "kube-api-access-nkgqb") pod "0ea73384-51d5-43b0-9413-4be04995aa31" (UID: "0ea73384-51d5-43b0-9413-4be04995aa31"). InnerVolumeSpecName "kube-api-access-nkgqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:20:32.153302 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.153282 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ea73384-51d5-43b0-9413-4be04995aa31" (UID: "0ea73384-51d5-43b0-9413-4be04995aa31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:20:32.252228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.252179 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea73384-51d5-43b0-9413-4be04995aa31-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:20:32.252228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.252223 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkgqb\" (UniqueName: \"kubernetes.io/projected/0ea73384-51d5-43b0-9413-4be04995aa31-kube-api-access-nkgqb\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:20:32.252228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.252239 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea73384-51d5-43b0-9413-4be04995aa31-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:20:32.297976 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.297940 2560 generic.go:356] "Generic (PLEG): container finished" podID="0ea73384-51d5-43b0-9413-4be04995aa31" containerID="d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c" exitCode=0 Jul 08 02:20:32.298151 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.297981 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" Jul 08 02:20:32.298151 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.298021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" event={"ID":"0ea73384-51d5-43b0-9413-4be04995aa31","Type":"ContainerDied","Data":"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c"} Jul 08 02:20:32.298151 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.298070 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh" event={"ID":"0ea73384-51d5-43b0-9413-4be04995aa31","Type":"ContainerDied","Data":"f1750d03a7080db34317e279fdd401a53802525a12a0d68db07d8a5d47ebb040"} Jul 08 02:20:32.298151 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.298092 2560 scope.go:117] "RemoveContainer" containerID="d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c" Jul 08 02:20:32.307201 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.307181 2560 scope.go:117] "RemoveContainer" containerID="d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c" Jul 08 02:20:32.307490 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:20:32.307469 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c\": container with ID starting with d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c not found: ID does not exist" containerID="d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c" Jul 08 02:20:32.307554 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.307498 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c"} err="failed to get container status \"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c\": rpc error: code = NotFound desc = could not find container \"d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c\": container with ID starting with d797c4189afe8f05e23eb5e58f9b9360dc51dcd244106bf6ca78b7005476784c not found: ID does not exist" Jul 08 02:20:32.320894 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.320866 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:20:32.324824 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:32.324796 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c26aa-6d55777874-d9zmh"] Jul 08 02:20:33.030246 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:20:33.030214 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea73384-51d5-43b0-9413-4be04995aa31" path="/var/lib/kubelet/pods/0ea73384-51d5-43b0-9413-4be04995aa31/volumes" Jul 08 02:21:14.874753 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:14.874712 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:21:14.875397 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:14.874986 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" podUID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" containerName="maas-api" containerID="cri-o://4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1" gracePeriod=30 Jul 08 02:21:15.129611 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.129543 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:21:15.230808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.230767 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca\") pod \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " Jul 08 02:21:15.231043 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.230859 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls\") pod \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " Jul 08 02:21:15.231043 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.230892 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgjgh\" (UniqueName: \"kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh\") pod \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\" (UID: \"e5c7e2c6-6e53-4484-8a90-4a5a215118fd\") " Jul 08 02:21:15.231254 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.231229 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5c7e2c6-6e53-4484-8a90-4a5a215118fd" (UID: "e5c7e2c6-6e53-4484-8a90-4a5a215118fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:21:15.233058 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.233029 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5c7e2c6-6e53-4484-8a90-4a5a215118fd" (UID: "e5c7e2c6-6e53-4484-8a90-4a5a215118fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:21:15.233170 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.233097 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh" (OuterVolumeSpecName: "kube-api-access-mgjgh") pod "e5c7e2c6-6e53-4484-8a90-4a5a215118fd" (UID: "e5c7e2c6-6e53-4484-8a90-4a5a215118fd"). InnerVolumeSpecName "kube-api-access-mgjgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:21:15.332228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.332187 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:15.332228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.332222 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:15.332228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.332231 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgjgh\" (UniqueName: \"kubernetes.io/projected/e5c7e2c6-6e53-4484-8a90-4a5a215118fd-kube-api-access-mgjgh\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:15.526808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.526695 2560 generic.go:356] "Generic (PLEG): container finished" podID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" containerID="4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1" exitCode=0 Jul 08 02:21:15.526808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.526746 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" Jul 08 02:21:15.526808 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.526784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" event={"ID":"e5c7e2c6-6e53-4484-8a90-4a5a215118fd","Type":"ContainerDied","Data":"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1"} Jul 08 02:21:15.527110 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.526870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h" event={"ID":"e5c7e2c6-6e53-4484-8a90-4a5a215118fd","Type":"ContainerDied","Data":"6a2c1dd43dab569b874f5911d8747da5e77d0c317f069eef7e8543f94d756823"} Jul 08 02:21:15.527110 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.526901 2560 scope.go:117] "RemoveContainer" containerID="4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1" Jul 08 02:21:15.536352 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.536336 2560 scope.go:117] "RemoveContainer" containerID="4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1" Jul 08 02:21:15.536615 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:15.536597 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1\": container with ID starting with 4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1 not found: ID does not exist" containerID="4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1" Jul 08 02:21:15.536667 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.536623 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1"} err="failed to get container status \"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1\": rpc error: code = NotFound desc = could not find container \"4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1\": container with ID starting with 4298f4b8d0e951b57479acddd0ca2b331367797e86d5629c5f57099677617be1 not found: ID does not exist" Jul 08 02:21:15.549147 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.549119 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:21:15.551011 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:15.550991 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-219487-5d46f76c9b-qj98h"] Jul 08 02:21:17.029266 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:17.029218 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" path="/var/lib/kubelet/pods/e5c7e2c6-6e53-4484-8a90-4a5a215118fd/volumes" Jul 08 02:21:24.608714 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.608672 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:24.609196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609153 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06cb567-1932-495c-89fc-56abd6ac58c9" containerName="maas-api" Jul 08 02:21:24.609196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609167 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06cb567-1932-495c-89fc-56abd6ac58c9" containerName="maas-api" Jul 08 02:21:24.609196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609185 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" containerName="maas-api" Jul 08 02:21:24.609196 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609191 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" containerName="maas-api" Jul 08 02:21:24.609335 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609202 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea73384-51d5-43b0-9413-4be04995aa31" containerName="maas-api" Jul 08 02:21:24.609335 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609207 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea73384-51d5-43b0-9413-4be04995aa31" containerName="maas-api" Jul 08 02:21:24.609335 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609263 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a06cb567-1932-495c-89fc-56abd6ac58c9" containerName="maas-api" Jul 08 02:21:24.609335 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609270 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c7e2c6-6e53-4484-8a90-4a5a215118fd" containerName="maas-api" Jul 08 02:21:24.609335 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.609276 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea73384-51d5-43b0-9413-4be04995aa31" containerName="maas-api" Jul 08 02:21:24.612428 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.612411 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.615241 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.615217 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:21:24.615373 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.615225 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:21:24.616254 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.616239 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:21:24.616341 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.616245 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-645239-serving-cert\"" Jul 08 02:21:24.623020 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.622996 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:24.717819 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.717781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7wrc\" (UniqueName: \"kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.718009 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.717844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.718009 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.717961 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.818668 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.818629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.818923 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.818702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.818923 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.818743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7wrc\" (UniqueName: \"kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.818923 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:24.818790 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-645239-serving-cert: secret "maas-api-e2e-api-a-645239-serving-cert" not found Jul 08 02:21:24.818923 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:24.818895 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls podName:042cad53-e919-4eef-9d3b-afa2878f5762 nodeName:}" failed. No retries permitted until 2026-07-08 02:21:25.318877658 +0000 UTC m=+2316.773433281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls") pod "maas-api-e2e-api-a-645239-647c6c878-hsrf8" (UID: "042cad53-e919-4eef-9d3b-afa2878f5762") : secret "maas-api-e2e-api-a-645239-serving-cert" not found Jul 08 02:21:24.819471 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.819445 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:24.827432 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:24.827410 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7wrc\" (UniqueName: \"kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:25.323323 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:25.323287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:25.325727 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:25.325695 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") pod \"maas-api-e2e-api-a-645239-647c6c878-hsrf8\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:25.523269 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:25.523232 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:25.660797 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:25.660767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:25.663062 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:21:25.663031 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042cad53_e919_4eef_9d3b_afa2878f5762.slice/crio-c87975c914f3111e5c47931e7ada19b3727cb1b76be4859c6d52280dfdaf122b WatchSource:0}: Error finding container c87975c914f3111e5c47931e7ada19b3727cb1b76be4859c6d52280dfdaf122b: Status 404 returned error can't find the container with id c87975c914f3111e5c47931e7ada19b3727cb1b76be4859c6d52280dfdaf122b Jul 08 02:21:25.664741 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:25.664722 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 02:21:26.588300 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:26.588267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" event={"ID":"042cad53-e919-4eef-9d3b-afa2878f5762","Type":"ContainerStarted","Data":"111a973f6728fcf9609678e4b04d22ae482216800fee6ec24eb80439bbca42e5"} Jul 08 02:21:26.588484 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:26.588318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" event={"ID":"042cad53-e919-4eef-9d3b-afa2878f5762","Type":"ContainerStarted","Data":"c87975c914f3111e5c47931e7ada19b3727cb1b76be4859c6d52280dfdaf122b"} Jul 08 02:21:26.588484 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:26.588350 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:26.605527 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:26.605474 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" podStartSLOduration=2.191283288 podStartE2EDuration="2.605456263s" totalImagesPullingTime="414.172975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:21:26.604255286 +0000 UTC m=+2318.058810934" watchObservedRunningTime="2026-07-08 02:21:26.605456263 +0000 UTC m=+2318.060011907" Jul 08 02:21:32.244055 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.244012 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:32.247542 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.247517 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.250113 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.250088 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1df4d7-serving-cert\"" Jul 08 02:21:32.257893 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.257860 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:32.387046 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.387003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2cc\" (UniqueName: \"kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.387253 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.387081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.387253 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.387156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.488110 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.488067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.488286 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.488124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2cc\" (UniqueName: \"kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.488286 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.488172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.488377 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:32.488298 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-serving-cert: secret "maas-api-e2e-api-b-1df4d7-serving-cert" not found Jul 08 02:21:32.488377 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:32.488365 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls podName:a96b38ca-9bba-4bb6-9933-2656371c36e8 nodeName:}" failed. No retries permitted until 2026-07-08 02:21:32.988344029 +0000 UTC m=+2324.442899652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls") pod "maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" (UID: "a96b38ca-9bba-4bb6-9933-2656371c36e8") : secret "maas-api-e2e-api-b-1df4d7-serving-cert" not found Jul 08 02:21:32.488762 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.488742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.499476 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.499413 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2cc\" (UniqueName: \"kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.598798 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.598766 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:32.993115 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.993067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:32.995549 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:32.995529 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") pod \"maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:33.159234 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:33.159197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:33.300731 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:33.300706 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:33.302777 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:21:33.302748 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96b38ca_9bba_4bb6_9933_2656371c36e8.slice/crio-8f18453a07c5debecb271826990d14a9cf5821b4ddfb83f7e387553da403f239 WatchSource:0}: Error finding container 8f18453a07c5debecb271826990d14a9cf5821b4ddfb83f7e387553da403f239: Status 404 returned error can't find the container with id 8f18453a07c5debecb271826990d14a9cf5821b4ddfb83f7e387553da403f239 Jul 08 02:21:33.625801 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:33.625768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" event={"ID":"a96b38ca-9bba-4bb6-9933-2656371c36e8","Type":"ContainerStarted","Data":"8f18453a07c5debecb271826990d14a9cf5821b4ddfb83f7e387553da403f239"} Jul 08 02:21:34.631852 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:34.631794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" event={"ID":"a96b38ca-9bba-4bb6-9933-2656371c36e8","Type":"ContainerStarted","Data":"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e"} Jul 08 02:21:34.632237 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:34.631879 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:34.651061 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:34.651006 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" podStartSLOduration=2.321794147 podStartE2EDuration="2.650984909s" totalImagesPullingTime="329.190762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:21:34.650147824 +0000 UTC m=+2326.104703469" watchObservedRunningTime="2026-07-08 02:21:34.650984909 +0000 UTC m=+2326.105540555" Jul 08 02:21:40.643324 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:40.643288 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:45.527893 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.527856 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:45.528495 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.528439 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" podUID="042cad53-e919-4eef-9d3b-afa2878f5762" containerName="maas-api" containerID="cri-o://111a973f6728fcf9609678e4b04d22ae482216800fee6ec24eb80439bbca42e5" gracePeriod=30 Jul 08 02:21:45.690292 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.690260 2560 generic.go:356] "Generic (PLEG): container finished" podID="042cad53-e919-4eef-9d3b-afa2878f5762" containerID="111a973f6728fcf9609678e4b04d22ae482216800fee6ec24eb80439bbca42e5" exitCode=0 Jul 08 02:21:45.690455 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.690304 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" event={"ID":"042cad53-e919-4eef-9d3b-afa2878f5762","Type":"ContainerDied","Data":"111a973f6728fcf9609678e4b04d22ae482216800fee6ec24eb80439bbca42e5"} Jul 08 02:21:45.781790 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.781729 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:45.914456 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.914413 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7wrc\" (UniqueName: \"kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc\") pod \"042cad53-e919-4eef-9d3b-afa2878f5762\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " Jul 08 02:21:45.914636 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.914521 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") pod \"042cad53-e919-4eef-9d3b-afa2878f5762\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " Jul 08 02:21:45.914636 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.914573 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca\") pod \"042cad53-e919-4eef-9d3b-afa2878f5762\" (UID: \"042cad53-e919-4eef-9d3b-afa2878f5762\") " Jul 08 02:21:45.915004 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.914973 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "042cad53-e919-4eef-9d3b-afa2878f5762" (UID: "042cad53-e919-4eef-9d3b-afa2878f5762"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:21:45.916613 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.916590 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "042cad53-e919-4eef-9d3b-afa2878f5762" (UID: "042cad53-e919-4eef-9d3b-afa2878f5762"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:21:45.916675 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:45.916612 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc" (OuterVolumeSpecName: "kube-api-access-j7wrc") pod "042cad53-e919-4eef-9d3b-afa2878f5762" (UID: "042cad53-e919-4eef-9d3b-afa2878f5762"). InnerVolumeSpecName "kube-api-access-j7wrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:21:46.015883 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.015821 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/042cad53-e919-4eef-9d3b-afa2878f5762-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:46.015883 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.015879 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/042cad53-e919-4eef-9d3b-afa2878f5762-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:46.015883 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.015888 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7wrc\" (UniqueName: \"kubernetes.io/projected/042cad53-e919-4eef-9d3b-afa2878f5762-kube-api-access-j7wrc\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:46.696092 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.696065 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" Jul 08 02:21:46.696480 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.696104 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8" event={"ID":"042cad53-e919-4eef-9d3b-afa2878f5762","Type":"ContainerDied","Data":"c87975c914f3111e5c47931e7ada19b3727cb1b76be4859c6d52280dfdaf122b"} Jul 08 02:21:46.696480 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.696166 2560 scope.go:117] "RemoveContainer" containerID="111a973f6728fcf9609678e4b04d22ae482216800fee6ec24eb80439bbca42e5" Jul 08 02:21:46.719538 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.719512 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:46.722931 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:46.722909 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-645239-647c6c878-hsrf8"] Jul 08 02:21:47.029288 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:47.029209 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042cad53-e919-4eef-9d3b-afa2878f5762" path="/var/lib/kubelet/pods/042cad53-e919-4eef-9d3b-afa2878f5762/volumes" Jul 08 02:21:52.402828 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.402793 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:52.403334 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.403046 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" podUID="a96b38ca-9bba-4bb6-9933-2656371c36e8" containerName="maas-api" containerID="cri-o://2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e" gracePeriod=30 Jul 08 02:21:52.659271 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.659202 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:52.673634 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.673590 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") pod \"a96b38ca-9bba-4bb6-9933-2656371c36e8\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " Jul 08 02:21:52.674036 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.673643 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g2cc\" (UniqueName: \"kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc\") pod \"a96b38ca-9bba-4bb6-9933-2656371c36e8\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " Jul 08 02:21:52.674036 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.673669 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca\") pod \"a96b38ca-9bba-4bb6-9933-2656371c36e8\" (UID: \"a96b38ca-9bba-4bb6-9933-2656371c36e8\") " Jul 08 02:21:52.674546 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.674298 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a96b38ca-9bba-4bb6-9933-2656371c36e8" (UID: "a96b38ca-9bba-4bb6-9933-2656371c36e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:21:52.676581 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.676554 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a96b38ca-9bba-4bb6-9933-2656371c36e8" (UID: "a96b38ca-9bba-4bb6-9933-2656371c36e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:21:52.676956 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.676927 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc" (OuterVolumeSpecName: "kube-api-access-6g2cc") pod "a96b38ca-9bba-4bb6-9933-2656371c36e8" (UID: "a96b38ca-9bba-4bb6-9933-2656371c36e8"). InnerVolumeSpecName "kube-api-access-6g2cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:21:52.727374 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.727330 2560 generic.go:356] "Generic (PLEG): container finished" podID="a96b38ca-9bba-4bb6-9933-2656371c36e8" containerID="2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e" exitCode=0 Jul 08 02:21:52.727514 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.727378 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" Jul 08 02:21:52.727514 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.727412 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" event={"ID":"a96b38ca-9bba-4bb6-9933-2656371c36e8","Type":"ContainerDied","Data":"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e"} Jul 08 02:21:52.727514 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.727472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj" event={"ID":"a96b38ca-9bba-4bb6-9933-2656371c36e8","Type":"ContainerDied","Data":"8f18453a07c5debecb271826990d14a9cf5821b4ddfb83f7e387553da403f239"} Jul 08 02:21:52.727514 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.727500 2560 scope.go:117] "RemoveContainer" containerID="2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e" Jul 08 02:21:52.736596 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.736579 2560 scope.go:117] "RemoveContainer" containerID="2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e" Jul 08 02:21:52.736826 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:21:52.736808 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e\": container with ID starting with 2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e not found: ID does not exist" containerID="2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e" Jul 08 02:21:52.736898 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.736847 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e"} err="failed to get container status \"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e\": rpc error: code = NotFound desc = could not find container \"2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e\": container with ID starting with 2af1af43459f2917d056d593b999800c7a2afd3803e8b8dbecbb33719a7c335e not found: ID does not exist" Jul 08 02:21:52.748402 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.748379 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:52.750327 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.750307 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1df4d7-6476666f46-nlkzj"] Jul 08 02:21:52.775174 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.775147 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a96b38ca-9bba-4bb6-9933-2656371c36e8-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:52.775174 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.775174 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g2cc\" (UniqueName: \"kubernetes.io/projected/a96b38ca-9bba-4bb6-9933-2656371c36e8-kube-api-access-6g2cc\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:52.775356 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:52.775184 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a96b38ca-9bba-4bb6-9933-2656371c36e8-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:21:53.029608 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:21:53.029528 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a96b38ca-9bba-4bb6-9933-2656371c36e8" path="/var/lib/kubelet/pods/a96b38ca-9bba-4bb6-9933-2656371c36e8/volumes" Jul 08 02:22:02.384662 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.384623 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:22:02.385067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.384977 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96b38ca-9bba-4bb6-9933-2656371c36e8" containerName="maas-api" Jul 08 02:22:02.385067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.384989 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96b38ca-9bba-4bb6-9933-2656371c36e8" containerName="maas-api" Jul 08 02:22:02.385067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.385013 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042cad53-e919-4eef-9d3b-afa2878f5762" containerName="maas-api" Jul 08 02:22:02.385067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.385018 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="042cad53-e919-4eef-9d3b-afa2878f5762" containerName="maas-api" Jul 08 02:22:02.385067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.385066 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="042cad53-e919-4eef-9d3b-afa2878f5762" containerName="maas-api" Jul 08 02:22:02.385233 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.385076 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a96b38ca-9bba-4bb6-9933-2656371c36e8" containerName="maas-api" Jul 08 02:22:02.387997 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.387979 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.391667 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.391643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7bwwv\"" Jul 08 02:22:02.392128 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.392108 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 02:22:02.392334 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.392320 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-4ea2af-serving-cert\"" Jul 08 02:22:02.394444 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.394426 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 02:22:02.415371 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.415338 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:22:02.467990 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.467955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.468179 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.468019 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps8fj\" (UniqueName: \"kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.468179 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.468125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.569181 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.569140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.569338 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.569195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps8fj\" (UniqueName: \"kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.569338 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.569255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.569431 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:02.569365 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-serving-cert: secret "maas-api-e2e-shared-a-4ea2af-serving-cert" not found Jul 08 02:22:02.569470 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:02.569441 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls podName:50388499-0543-4f3e-8d9e-e62881c28cd3 nodeName:}" failed. No retries permitted until 2026-07-08 02:22:03.069421252 +0000 UTC m=+2354.523976876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls") pod "maas-api-e2e-shared-a-4ea2af-998859f69-47btk" (UID: "50388499-0543-4f3e-8d9e-e62881c28cd3") : secret "maas-api-e2e-shared-a-4ea2af-serving-cert" not found Jul 08 02:22:02.569820 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.569790 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:02.580278 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:02.580249 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps8fj\" (UniqueName: \"kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:03.073423 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:03.073384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:03.073622 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:03.073533 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-serving-cert: secret "maas-api-e2e-shared-a-4ea2af-serving-cert" not found Jul 08 02:22:03.073622 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:03.073599 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls podName:50388499-0543-4f3e-8d9e-e62881c28cd3 nodeName:}" failed. No retries permitted until 2026-07-08 02:22:04.073582738 +0000 UTC m=+2355.528138361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls") pod "maas-api-e2e-shared-a-4ea2af-998859f69-47btk" (UID: "50388499-0543-4f3e-8d9e-e62881c28cd3") : secret "maas-api-e2e-shared-a-4ea2af-serving-cert" not found Jul 08 02:22:04.083857 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:04.083794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:04.086253 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:04.086228 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ea2af-998859f69-47btk\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:04.198565 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:04.198511 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:04.334351 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:04.334324 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:22:04.336433 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:22:04.336408 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50388499_0543_4f3e_8d9e_e62881c28cd3.slice/crio-be27d076d4395c0c9bc4c42a8c82edd9e27aaf439d4ccffbea9448dd6ab10f05 WatchSource:0}: Error finding container be27d076d4395c0c9bc4c42a8c82edd9e27aaf439d4ccffbea9448dd6ab10f05: Status 404 returned error can't find the container with id be27d076d4395c0c9bc4c42a8c82edd9e27aaf439d4ccffbea9448dd6ab10f05 Jul 08 02:22:04.791248 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:04.791220 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" event={"ID":"50388499-0543-4f3e-8d9e-e62881c28cd3","Type":"ContainerStarted","Data":"be27d076d4395c0c9bc4c42a8c82edd9e27aaf439d4ccffbea9448dd6ab10f05"} Jul 08 02:22:05.798809 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:05.798770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" event={"ID":"50388499-0543-4f3e-8d9e-e62881c28cd3","Type":"ContainerStarted","Data":"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5"} Jul 08 02:22:05.799228 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:05.798888 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:05.816826 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:05.816774 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" podStartSLOduration=3.414776425 podStartE2EDuration="3.816756112s" totalImagesPullingTime="401.979687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:22:05.814351408 +0000 UTC m=+2357.268907053" watchObservedRunningTime="2026-07-08 02:22:05.816756112 +0000 UTC m=+2357.271311757" Jul 08 02:22:10.364151 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.364112 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:22:10.367887 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.367860 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.370218 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.370192 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b8b505-serving-cert\"" Jul 08 02:22:10.378036 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.378011 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:22:10.441936 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.441898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.442137 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.441965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.442137 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.442051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lxhm\" (UniqueName: \"kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.543501 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.543444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.543501 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.543516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lxhm\" (UniqueName: \"kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.543783 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.543572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.543783 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:10.543664 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-serving-cert: secret "maas-api-e2e-shared-b-b8b505-serving-cert" not found Jul 08 02:22:10.543783 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:22:10.543729 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls podName:415f72f0-40d4-4076-90d8-7e32d1379d03 nodeName:}" failed. No retries permitted until 2026-07-08 02:22:11.043712849 +0000 UTC m=+2362.498268473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls") pod "maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" (UID: "415f72f0-40d4-4076-90d8-7e32d1379d03") : secret "maas-api-e2e-shared-b-b8b505-serving-cert" not found Jul 08 02:22:10.544272 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.544247 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:10.564752 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:10.564713 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lxhm\" (UniqueName: \"kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:11.048552 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.048521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:11.050937 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.050907 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") pod \"maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:11.279382 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.279343 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:11.416125 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.416088 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:22:11.809446 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.809420 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:22:11.829885 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:11.829845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" event={"ID":"415f72f0-40d4-4076-90d8-7e32d1379d03","Type":"ContainerStarted","Data":"03c716e2d9bb86e777148f0dfa318e4ba2880b04d5275bc5db087be84071c1ef"} Jul 08 02:22:12.836354 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:12.836322 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" event={"ID":"415f72f0-40d4-4076-90d8-7e32d1379d03","Type":"ContainerStarted","Data":"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969"} Jul 08 02:22:12.836755 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:12.836413 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:22:12.854784 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:12.854721 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" podStartSLOduration=2.480137928 podStartE2EDuration="2.854697524s" totalImagesPullingTime="374.559596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 02:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:22:12.851634014 +0000 UTC m=+2364.306189659" watchObservedRunningTime="2026-07-08 02:22:12.854697524 +0000 UTC m=+2364.309253184" Jul 08 02:22:18.847556 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:22:18.847527 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:25:20.433606 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.433567 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:25:20.436140 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.433821 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" podUID="50388499-0543-4f3e-8d9e-e62881c28cd3" containerName="maas-api" containerID="cri-o://c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5" gracePeriod=30 Jul 08 02:25:20.691135 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.691067 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:25:20.789236 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.789203 2560 generic.go:356] "Generic (PLEG): container finished" podID="50388499-0543-4f3e-8d9e-e62881c28cd3" containerID="c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5" exitCode=0 Jul 08 02:25:20.789417 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.789249 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" Jul 08 02:25:20.789417 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.789269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" event={"ID":"50388499-0543-4f3e-8d9e-e62881c28cd3","Type":"ContainerDied","Data":"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5"} Jul 08 02:25:20.789417 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.789310 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk" event={"ID":"50388499-0543-4f3e-8d9e-e62881c28cd3","Type":"ContainerDied","Data":"be27d076d4395c0c9bc4c42a8c82edd9e27aaf439d4ccffbea9448dd6ab10f05"} Jul 08 02:25:20.789417 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.789331 2560 scope.go:117] "RemoveContainer" containerID="c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5" Jul 08 02:25:20.798075 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.798055 2560 scope.go:117] "RemoveContainer" containerID="c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5" Jul 08 02:25:20.798323 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:25:20.798304 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5\": container with ID starting with c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5 not found: ID does not exist" containerID="c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5" Jul 08 02:25:20.798370 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.798332 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5"} err="failed to get container status \"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5\": rpc error: code = NotFound desc = could not find container \"c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5\": container with ID starting with c852152379f45d57a4504cac7e5916f0953393b799e958c70e05a866f9e6a5a5 not found: ID does not exist" Jul 08 02:25:20.816772 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.816747 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca\") pod \"50388499-0543-4f3e-8d9e-e62881c28cd3\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " Jul 08 02:25:20.816886 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.816781 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps8fj\" (UniqueName: \"kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj\") pod \"50388499-0543-4f3e-8d9e-e62881c28cd3\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " Jul 08 02:25:20.816886 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.816827 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") pod \"50388499-0543-4f3e-8d9e-e62881c28cd3\" (UID: \"50388499-0543-4f3e-8d9e-e62881c28cd3\") " Jul 08 02:25:20.817115 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.817084 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50388499-0543-4f3e-8d9e-e62881c28cd3" (UID: "50388499-0543-4f3e-8d9e-e62881c28cd3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:25:20.818864 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.818815 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50388499-0543-4f3e-8d9e-e62881c28cd3" (UID: "50388499-0543-4f3e-8d9e-e62881c28cd3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:25:20.818959 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.818875 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj" (OuterVolumeSpecName: "kube-api-access-ps8fj") pod "50388499-0543-4f3e-8d9e-e62881c28cd3" (UID: "50388499-0543-4f3e-8d9e-e62881c28cd3"). InnerVolumeSpecName "kube-api-access-ps8fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:25:20.918300 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.918258 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50388499-0543-4f3e-8d9e-e62881c28cd3-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:20.918300 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.918292 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps8fj\" (UniqueName: \"kubernetes.io/projected/50388499-0543-4f3e-8d9e-e62881c28cd3-kube-api-access-ps8fj\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:20.918300 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:20.918303 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50388499-0543-4f3e-8d9e-e62881c28cd3-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:21.104846 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:21.104795 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:25:21.109068 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:21.109044 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ea2af-998859f69-47btk"] Jul 08 02:25:23.029594 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:23.029557 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50388499-0543-4f3e-8d9e-e62881c28cd3" path="/var/lib/kubelet/pods/50388499-0543-4f3e-8d9e-e62881c28cd3/volumes" Jul 08 02:25:27.302490 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.302446 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:25:27.302987 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.302670 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" podUID="415f72f0-40d4-4076-90d8-7e32d1379d03" containerName="maas-api" containerID="cri-o://8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969" gracePeriod=30 Jul 08 02:25:27.546636 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.546606 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:25:27.678311 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.678274 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca\") pod \"415f72f0-40d4-4076-90d8-7e32d1379d03\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " Jul 08 02:25:27.678311 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.678316 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lxhm\" (UniqueName: \"kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm\") pod \"415f72f0-40d4-4076-90d8-7e32d1379d03\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " Jul 08 02:25:27.678598 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.678368 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") pod \"415f72f0-40d4-4076-90d8-7e32d1379d03\" (UID: \"415f72f0-40d4-4076-90d8-7e32d1379d03\") " Jul 08 02:25:27.678716 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.678683 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "415f72f0-40d4-4076-90d8-7e32d1379d03" (UID: "415f72f0-40d4-4076-90d8-7e32d1379d03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 02:25:27.680491 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.680459 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "415f72f0-40d4-4076-90d8-7e32d1379d03" (UID: "415f72f0-40d4-4076-90d8-7e32d1379d03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 02:25:27.680598 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.680553 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm" (OuterVolumeSpecName: "kube-api-access-6lxhm") pod "415f72f0-40d4-4076-90d8-7e32d1379d03" (UID: "415f72f0-40d4-4076-90d8-7e32d1379d03"). InnerVolumeSpecName "kube-api-access-6lxhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 02:25:27.779171 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.779123 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lxhm\" (UniqueName: \"kubernetes.io/projected/415f72f0-40d4-4076-90d8-7e32d1379d03-kube-api-access-6lxhm\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:27.779171 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.779165 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/415f72f0-40d4-4076-90d8-7e32d1379d03-maas-api-tls\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:27.779171 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.779179 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/415f72f0-40d4-4076-90d8-7e32d1379d03-openshift-service-ca\") on node \"ip-10-0-136-194.ec2.internal\" DevicePath \"\"" Jul 08 02:25:27.825118 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.825072 2560 generic.go:356] "Generic (PLEG): container finished" podID="415f72f0-40d4-4076-90d8-7e32d1379d03" containerID="8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969" exitCode=0 Jul 08 02:25:27.825290 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.825170 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" Jul 08 02:25:27.825290 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.825167 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" event={"ID":"415f72f0-40d4-4076-90d8-7e32d1379d03","Type":"ContainerDied","Data":"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969"} Jul 08 02:25:27.825290 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.825262 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw" event={"ID":"415f72f0-40d4-4076-90d8-7e32d1379d03","Type":"ContainerDied","Data":"03c716e2d9bb86e777148f0dfa318e4ba2880b04d5275bc5db087be84071c1ef"} Jul 08 02:25:27.825439 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.825294 2560 scope.go:117] "RemoveContainer" containerID="8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969" Jul 08 02:25:27.834999 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.834971 2560 scope.go:117] "RemoveContainer" containerID="8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969" Jul 08 02:25:27.835324 ip-10-0-136-194 kubenswrapper[2560]: E0708 02:25:27.835305 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969\": container with ID starting with 8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969 not found: ID does not exist" containerID="8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969" Jul 08 02:25:27.835403 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.835339 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969"} err="failed to get container status \"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969\": rpc error: code = NotFound desc = could not find container \"8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969\": container with ID starting with 8f101aa7320d8d8433f37ff2da18d327103d3c47416962de325f4230bedba969 not found: ID does not exist" Jul 08 02:25:27.846167 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.846134 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:25:27.852254 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:27.852226 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b8b505-75bfc75ff7-6q4dw"] Jul 08 02:25:29.029342 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:29.029301 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="415f72f0-40d4-4076-90d8-7e32d1379d03" path="/var/lib/kubelet/pods/415f72f0-40d4-4076-90d8-7e32d1379d03/volumes" Jul 08 02:25:39.054814 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:39.054780 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4pmpx_624e05de-7460-4e39-8ecd-549a27d15e40/manager/0.log" Jul 08 02:25:39.429690 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:39.429613 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bbd955f5-c8cm4_d984c7b0-936c-4728-8cbd-be31e836a65b/copy-manifests/0.log" Jul 08 02:25:39.436592 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:39.436560 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bbd955f5-c8cm4_d984c7b0-936c-4728-8cbd-be31e836a65b/manager/0.log" Jul 08 02:25:40.255067 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.255031 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/util/0.log" Jul 08 02:25:40.261941 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.261919 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/pull/0.log" Jul 08 02:25:40.268116 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.268095 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/extract/0.log" Jul 08 02:25:40.384236 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.384206 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/util/0.log" Jul 08 02:25:40.393489 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.393461 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/pull/0.log" Jul 08 02:25:40.400521 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.400494 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/extract/0.log" Jul 08 02:25:40.510102 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.510029 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/util/0.log" Jul 08 02:25:40.517204 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.517178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/pull/0.log" Jul 08 02:25:40.523647 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.523623 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/extract/0.log" Jul 08 02:25:40.638133 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.638105 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/util/0.log" Jul 08 02:25:40.644590 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.644566 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/pull/0.log" Jul 08 02:25:40.651363 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:40.651335 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/extract/0.log" Jul 08 02:25:41.354013 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:41.353982 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tgrph_b2921e97-2aba-41fd-8876-cb58ef622135/manager/0.log" Jul 08 02:25:42.059924 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:42.059892 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v6cjh_acd70d71-741b-482d-bc0d-a896bfe77652/discovery/0.log" Jul 08 02:25:43.243200 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:43.243167 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-kdqrt_823205e7-d413-4eff-85a0-6c854c7d6a7f/main/0.log" Jul 08 02:25:43.357153 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:43.357124 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-swxz8_9c746050-af7b-4bdb-9af2-83d7ddb81609/main/0.log" Jul 08 02:25:51.031176 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:51.031140 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dscg2_6e5523b0-6b1c-447e-b12c-3b1170259168/global-pull-secret-syncer/0.log" Jul 08 02:25:51.200096 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:51.200060 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x4wnz_f1d94f02-e63b-4528-a166-41f84f70bf87/konnectivity-agent/0.log" Jul 08 02:25:51.242827 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:51.242791 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-194.ec2.internal_d1387eef4cf6fa2f7fb83b7ff4442348/haproxy/0.log" Jul 08 02:25:56.279293 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.279258 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/extract/0.log" Jul 08 02:25:56.302578 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.302549 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/util/0.log" Jul 08 02:25:56.330274 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.330247 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jb42p_31220df7-bcde-48b4-a1b4-1a61ffb749c6/pull/0.log" Jul 08 02:25:56.358817 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.358790 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/extract/0.log" Jul 08 02:25:56.382675 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.382646 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/util/0.log" Jul 08 02:25:56.407933 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.407900 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sb97p_fd545210-0ccc-4eef-b584-d1c6aade3ae2/pull/0.log" Jul 08 02:25:56.439824 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.439789 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/extract/0.log" Jul 08 02:25:56.467238 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.467208 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/util/0.log" Jul 08 02:25:56.492994 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.492960 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dmn7m_559b81e8-b609-403f-9e0a-cecf99b401ca/pull/0.log" Jul 08 02:25:56.522544 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.522516 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/extract/0.log" Jul 08 02:25:56.546355 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.546276 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/util/0.log" Jul 08 02:25:56.567670 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:56.567648 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef158gkb_3acb6b09-2df9-4c29-9990-569c83b5997f/pull/0.log" Jul 08 02:25:57.194695 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:57.194658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tgrph_b2921e97-2aba-41fd-8876-cb58ef622135/manager/0.log" Jul 08 02:25:58.926473 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:58.926443 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/alertmanager/0.log" Jul 08 02:25:58.947449 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:58.947427 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/config-reloader/0.log" Jul 08 02:25:58.969791 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:58.969768 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/kube-rbac-proxy-web/0.log" Jul 08 02:25:58.995284 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:58.995255 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/kube-rbac-proxy/0.log" Jul 08 02:25:59.027200 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.027153 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/kube-rbac-proxy-metric/0.log" Jul 08 02:25:59.050654 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.050623 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/prom-label-proxy/0.log" Jul 08 02:25:59.072493 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.072463 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c759fae-ab4d-40d8-ba3f-096e30bc7a4a/init-config-reloader/0.log" Jul 08 02:25:59.244483 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.244401 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-n2rlt_8cddad98-596f-4423-b667-a980ad86c8f1/monitoring-plugin/0.log" Jul 08 02:25:59.369488 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.369461 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mb482_2951124e-c4b7-43e0-a74e-eb932dda57d0/node-exporter/0.log" Jul 08 02:25:59.410649 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.410620 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mb482_2951124e-c4b7-43e0-a74e-eb932dda57d0/kube-rbac-proxy/0.log" Jul 08 02:25:59.435977 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.435952 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mb482_2951124e-c4b7-43e0-a74e-eb932dda57d0/init-textfile/0.log" Jul 08 02:25:59.813596 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.813566 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-wg46q_0b2f2938-4c53-4322-93c3-5fddf586aa49/prometheus-operator/0.log" Jul 08 02:25:59.833111 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.833060 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-wg46q_0b2f2938-4c53-4322-93c3-5fddf586aa49/kube-rbac-proxy/0.log" Jul 08 02:25:59.907143 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.907117 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c89c99c66-wx6k4_5f231b5d-22a4-41d2-99ec-452315f1f165/telemeter-client/0.log" Jul 08 02:25:59.928799 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.928773 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c89c99c66-wx6k4_5f231b5d-22a4-41d2-99ec-452315f1f165/reload/0.log" Jul 08 02:25:59.955699 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:25:59.955675 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c89c99c66-wx6k4_5f231b5d-22a4-41d2-99ec-452315f1f165/kube-rbac-proxy/0.log" Jul 08 02:26:01.534301 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534263 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26"] Jul 08 02:26:01.534671 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534607 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="415f72f0-40d4-4076-90d8-7e32d1379d03" containerName="maas-api" Jul 08 02:26:01.534671 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534619 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="415f72f0-40d4-4076-90d8-7e32d1379d03" containerName="maas-api" Jul 08 02:26:01.534671 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534636 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50388499-0543-4f3e-8d9e-e62881c28cd3" containerName="maas-api" Jul 08 02:26:01.534671 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534641 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50388499-0543-4f3e-8d9e-e62881c28cd3" containerName="maas-api" Jul 08 02:26:01.534806 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534691 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50388499-0543-4f3e-8d9e-e62881c28cd3" containerName="maas-api" Jul 08 02:26:01.534806 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.534699 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="415f72f0-40d4-4076-90d8-7e32d1379d03" containerName="maas-api" Jul 08 02:26:01.537728 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.537711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.539927 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.539903 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qwgww\"/\"openshift-service-ca.crt\"" Jul 08 02:26:01.540208 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.540190 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qwgww\"/\"default-dockercfg-pgwwr\"" Jul 08 02:26:01.540793 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.540780 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qwgww\"/\"kube-root-ca.crt\"" Jul 08 02:26:01.548641 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.548620 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26"] Jul 08 02:26:01.575488 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.575454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-lib-modules\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.575651 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.575499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-proc\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.575651 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.575580 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-podres\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.575651 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.575622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tmhq\" (UniqueName: \"kubernetes.io/projected/422acdf3-74b8-4674-b548-4a3a9eabef60-kube-api-access-2tmhq\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.575651 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.575643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-sys\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676317 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-proc\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676469 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-podres\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676469 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmhq\" (UniqueName: \"kubernetes.io/projected/422acdf3-74b8-4674-b548-4a3a9eabef60-kube-api-access-2tmhq\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676469 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-sys\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676469 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676403 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-proc\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676469 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676464 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-sys\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676673 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676501 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-podres\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676673 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-lib-modules\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.676673 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.676589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422acdf3-74b8-4674-b548-4a3a9eabef60-lib-modules\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.684576 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.684543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tmhq\" (UniqueName: \"kubernetes.io/projected/422acdf3-74b8-4674-b548-4a3a9eabef60-kube-api-access-2tmhq\") pod \"perf-node-gather-daemonset-p5n26\" (UID: \"422acdf3-74b8-4674-b548-4a3a9eabef60\") " pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.848612 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.848561 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:01.982906 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:01.982875 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26"] Jul 08 02:26:01.986009 ip-10-0-136-194 kubenswrapper[2560]: W0708 02:26:01.985965 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod422acdf3_74b8_4674_b548_4a3a9eabef60.slice/crio-90859d3dd1ce149c0133a1a70544d2244143df8977e8b6f6aff708fa005b90bb WatchSource:0}: Error finding container 90859d3dd1ce149c0133a1a70544d2244143df8977e8b6f6aff708fa005b90bb: Status 404 returned error can't find the container with id 90859d3dd1ce149c0133a1a70544d2244143df8977e8b6f6aff708fa005b90bb Jul 08 02:26:02.996533 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:02.996494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" event={"ID":"422acdf3-74b8-4674-b548-4a3a9eabef60","Type":"ContainerStarted","Data":"b44c007c9893a7249b43341d11476ffa0aa0fc55ca48f1600e13f8a2863ec3fc"} Jul 08 02:26:02.997052 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:02.996551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" event={"ID":"422acdf3-74b8-4674-b548-4a3a9eabef60","Type":"ContainerStarted","Data":"90859d3dd1ce149c0133a1a70544d2244143df8977e8b6f6aff708fa005b90bb"} Jul 08 02:26:02.997052 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:02.996583 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:03.014590 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:03.014542 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" podStartSLOduration=2.014525765 podStartE2EDuration="2.014525765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 02:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 02:26:03.013050178 +0000 UTC m=+2594.467605824" watchObservedRunningTime="2026-07-08 02:26:03.014525765 +0000 UTC m=+2594.469081409" Jul 08 02:26:04.089463 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:04.089435 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f58sk_6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58/dns/0.log" Jul 08 02:26:04.110706 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:04.110675 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f58sk_6dc4451f-f41c-4c4f-a1b6-cbe0b0728e58/kube-rbac-proxy/0.log" Jul 08 02:26:04.280720 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:04.280682 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nx92p_36edce4a-98a9-400a-89aa-2f7f731ffda4/dns-node-resolver/0.log" Jul 08 02:26:04.801053 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:04.801021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7568b7fc95-rg5bd_b725eeb9-8fd4-4e37-b9f4-38188cb7b8e3/registry/0.log" Jul 08 02:26:04.871072 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:04.871039 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sptm4_b727a716-41ca-45f3-8d69-50d2f5c98e57/node-ca/0.log" Jul 08 02:26:05.902976 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:05.902940 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v6cjh_acd70d71-741b-482d-bc0d-a896bfe77652/discovery/0.log" Jul 08 02:26:06.826367 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:06.826337 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-24xpb_09e2edcb-e53d-4a92-93d1-99c86bd8ce6e/serve-healthcheck-canary/0.log" Jul 08 02:26:07.814255 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:07.814223 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xfl5_73313f24-d6a1-4d65-86b6-9a667da7efa8/kube-rbac-proxy/0.log" Jul 08 02:26:07.837282 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:07.837254 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xfl5_73313f24-d6a1-4d65-86b6-9a667da7efa8/exporter/0.log" Jul 08 02:26:07.861937 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:07.861911 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7xfl5_73313f24-d6a1-4d65-86b6-9a667da7efa8/extractor/0.log" Jul 08 02:26:09.011265 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:09.011230 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qwgww/perf-node-gather-daemonset-p5n26" Jul 08 02:26:10.329647 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:10.329613 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4pmpx_624e05de-7460-4e39-8ecd-549a27d15e40/manager/0.log" Jul 08 02:26:10.523060 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:10.523027 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bbd955f5-c8cm4_d984c7b0-936c-4728-8cbd-be31e836a65b/manager/0.log" Jul 08 02:26:10.546026 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:10.545991 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bbd955f5-c8cm4_d984c7b0-936c-4728-8cbd-be31e836a65b/copy-manifests/0.log" Jul 08 02:26:11.682811 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:11.682775 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v6cjh_acd70d71-741b-482d-bc0d-a896bfe77652/discovery/0.log" Jul 08 02:26:11.844432 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:11.844343 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7769fc489f-qb5gs_e885f47e-16ba-4698-8284-e285b8d4ac67/manager/0.log" Jul 08 02:26:11.923056 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:11.923024 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5ccsd_5aac7308-c51a-4d77-bc97-0f601f9f9c27/openshift-lws-operator/0.log" Jul 08 02:26:13.110235 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:13.110204 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-v6cjh_acd70d71-741b-482d-bc0d-a896bfe77652/discovery/0.log" Jul 08 02:26:18.266724 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.266693 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/kube-multus-additional-cni-plugins/0.log" Jul 08 02:26:18.289671 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.289641 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/egress-router-binary-copy/0.log" Jul 08 02:26:18.312686 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.312658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/cni-plugins/0.log" Jul 08 02:26:18.334430 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.334407 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/bond-cni-plugin/0.log" Jul 08 02:26:18.357146 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.357119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/routeoverride-cni/0.log" Jul 08 02:26:18.377973 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.377946 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/whereabouts-cni-bincopy/0.log" Jul 08 02:26:18.399170 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.399141 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62ft8_2ce2ec86-6512-43a7-8d3f-37472143acec/whereabouts-cni/0.log" Jul 08 02:26:18.791296 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.791265 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vmcdq_74a03d78-5b1c-4abb-ab8b-d10f3c2ece0a/kube-multus/0.log" Jul 08 02:26:18.813567 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.813542 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84kdl_8266ac7a-4fa1-4a13-9ca3-49137e9303d0/network-metrics-daemon/0.log" Jul 08 02:26:18.834605 ip-10-0-136-194 kubenswrapper[2560]: I0708 02:26:18.834572 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84kdl_8266ac7a-4fa1-4a13-9ca3-49137e9303d0/kube-rbac-proxy/0.log"