Jul 13 21:19:58.517707 ip-10-0-131-109 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 21:19:58.517720 ip-10-0-131-109 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 21:19:58.517729 ip-10-0-131-109 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 21:19:58.518047 ip-10-0-131-109 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 21:20:08.553485 ip-10-0-131-109 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 21:20:08.553510 ip-10-0-131-109 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9434bfd0db164a9fba64699dd8a5dd54 -- Jul 13 21:22:03.839318 ip-10-0-131-109 systemd[1]: Starting Kubernetes Kubelet... Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.198930 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203963 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203978 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203982 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203985 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203989 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203993 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.203997 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.272181 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204000 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204005 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204008 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204011 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204014 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204017 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204020 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204023 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204026 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204029 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204032 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204035 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204038 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204041 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204044 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204047 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204050 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204053 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204056 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204063 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.275507 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204066 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204070 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204073 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204075 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204078 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204081 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204084 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204087 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204090 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204092 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204095 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204098 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204102 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204105 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204109 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204113 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204116 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204119 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204122 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.276711 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204126 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204130 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204133 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204137 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204140 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204146 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204150 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204153 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204157 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204160 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204163 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204166 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204169 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204172 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204175 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204178 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204181 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204184 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204187 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.278869 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204190 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204210 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204213 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204216 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204219 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204222 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204225 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204228 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204231 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204234 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204236 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204240 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204243 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204247 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204249 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204253 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204256 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204259 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204262 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204265 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.279879 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204268 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204270 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204273 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204276 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204278 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204281 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204285 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204288 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204291 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204294 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204296 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204299 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204302 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.204304 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205093 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205099 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205103 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205106 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205109 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.283538 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205115 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205119 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205122 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205126 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205130 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205133 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205137 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205140 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205142 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205145 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205148 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205151 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205153 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205157 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205161 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205164 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205166 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205169 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205172 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.284728 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205175 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205178 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205181 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205183 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205186 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205189 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205209 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205214 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205217 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205220 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205226 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205229 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205232 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205235 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205238 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205241 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205244 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205247 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205250 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205252 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.288886 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205256 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.286914 ip-10-0-131-109 systemd[1]: Started Kubernetes Kubelet. Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205259 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205262 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205264 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205267 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205270 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205272 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205275 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205278 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205280 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205283 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205287 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205289 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205292 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205295 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205297 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205300 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205303 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205306 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205309 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.290066 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205312 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205317 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205321 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205325 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205327 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205330 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205333 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205336 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205338 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205341 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205344 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205346 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205349 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205352 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205355 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205358 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205360 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205363 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205366 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.291032 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205369 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205375 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205378 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205381 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205383 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205386 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205389 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205392 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205394 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205397 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205400 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205403 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205406 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205408 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205412 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205416 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205837 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205843 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205847 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.294746 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205850 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205853 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205856 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205859 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205862 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205865 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205868 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205871 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205874 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205877 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205880 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205883 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205885 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205888 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205894 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205896 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205899 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205902 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205905 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205908 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.295668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205910 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205913 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205916 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205918 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205921 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205924 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205927 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205929 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205934 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205936 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205939 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205942 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205945 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205947 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205950 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205953 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205955 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205958 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205961 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205964 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.296574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205967 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205969 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205972 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205975 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205977 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205980 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205984 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205987 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205990 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205993 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205995 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.205998 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206001 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206003 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206006 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206009 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206011 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206014 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206017 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206019 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.298944 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206023 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206026 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206029 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206032 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206034 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206037 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206039 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206042 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206045 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206048 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206052 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206055 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206058 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206061 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206064 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206066 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206070 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206074 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206078 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.299860 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206081 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206084 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206087 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206090 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206093 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206096 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206099 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206102 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206104 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206107 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206110 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206113 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206115 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206119 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206122 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206125 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206128 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206237 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206246 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206253 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206258 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 21:22:04.302738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206263 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206267 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206271 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206281 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206285 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206288 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206292 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206296 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206300 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206303 2564 flags.go:64] FLAG: --cgroup-root="" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206308 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206311 2564 flags.go:64] FLAG: --client-ca-file="" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206314 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206317 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206323 2564 flags.go:64] FLAG: --cluster-domain="" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206326 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206329 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206333 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206336 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206340 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206343 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206347 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206350 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206354 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 13 21:22:04.303697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206359 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206362 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206365 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206369 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206373 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206376 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206379 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206383 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206386 2564 flags.go:64] FLAG: --enable-server="true" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206389 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206395 2564 flags.go:64] FLAG: --event-burst="100" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206399 2564 flags.go:64] FLAG: --event-qps="50" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206402 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206405 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206408 2564 flags.go:64] FLAG: --eviction-hard="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206413 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206416 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206419 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206424 2564 flags.go:64] FLAG: --eviction-soft="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206427 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206430 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206434 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206437 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206440 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206443 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 21:22:04.400918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206446 2564 flags.go:64] FLAG: --feature-gates="" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206450 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206453 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206457 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206460 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206463 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206467 2564 flags.go:64] FLAG: --help="false" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206471 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206474 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206477 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206481 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206485 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206488 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206491 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206494 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206497 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206500 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206503 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206507 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206510 2564 flags.go:64] FLAG: --kube-reserved="" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206513 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206516 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206519 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206523 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 21:22:04.401879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206526 2564 flags.go:64] FLAG: --lock-file="" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206530 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206533 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206536 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206542 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206546 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206549 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206552 2564 flags.go:64] FLAG: --logging-format="text" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206554 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206558 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206561 2564 flags.go:64] FLAG: --manifest-url="" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206564 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206569 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206573 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206577 2564 flags.go:64] FLAG: --max-pods="110" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206583 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206586 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206589 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206592 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206595 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206598 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206602 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206610 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206613 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206616 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 21:22:04.513311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206620 2564 flags.go:64] FLAG: --pod-cidr="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206623 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206629 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206632 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206636 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206639 2564 flags.go:64] FLAG: --port="10250" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206642 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206646 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-060ae8047c1f58fd7" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206651 2564 flags.go:64] FLAG: --qos-reserved="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206655 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206658 2564 flags.go:64] FLAG: --register-node="true" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206661 2564 flags.go:64] FLAG: --register-with-taints="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206665 2564 flags.go:64] FLAG: --registry-burst="10" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206667 2564 flags.go:64] FLAG: --registry-qps="5" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206671 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206674 2564 flags.go:64] FLAG: --reserved-memory="" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206678 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206681 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206684 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206688 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206691 2564 flags.go:64] FLAG: --runonce="false" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206694 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206697 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206701 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206705 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 21:22:04.537382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206708 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206711 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206714 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206718 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206721 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206724 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206727 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206730 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206733 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206736 2564 flags.go:64] FLAG: --system-cgroups="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206739 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206745 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206748 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206751 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206755 2564 flags.go:64] FLAG: --tls-min-version="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206762 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206765 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206768 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206771 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206774 2564 flags.go:64] FLAG: --v="2" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206778 2564 flags.go:64] FLAG: --version="false" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206783 2564 flags.go:64] FLAG: --vmodule="" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206788 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.206791 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 21:22:04.538744 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206947 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206952 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206955 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206959 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206962 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206965 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206970 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206973 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206976 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206980 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206983 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206986 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206989 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206992 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206995 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.206998 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207000 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207003 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207006 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207009 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.540773 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207011 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207014 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207017 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207021 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207025 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207027 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207030 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207033 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207036 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207038 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207041 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207044 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207047 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207050 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207053 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207055 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207058 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207061 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207066 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207070 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.541693 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207073 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207076 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207079 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207082 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207085 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207088 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207091 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207094 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207097 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207099 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207102 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207105 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207108 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207111 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207114 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207118 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207122 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207125 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207128 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207131 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.542814 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207133 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207136 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207139 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207141 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207144 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207147 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207149 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207153 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207157 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207161 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207166 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207169 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207172 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207175 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207179 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207182 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207185 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207188 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.543737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207209 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207213 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207216 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207220 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207223 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207226 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207229 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207233 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207236 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207241 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207244 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207247 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207251 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207254 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207257 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207260 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207263 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207266 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207269 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207271 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.544646 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.207274 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.208001 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.215180 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.215212 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.215945 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.215957 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216019 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216026 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216031 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216034 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216037 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216040 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216043 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216047 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216050 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.546267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216054 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216058 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216061 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216066 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216070 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216073 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216076 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216079 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216083 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216085 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216088 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216091 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216094 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216097 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216100 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216103 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216106 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216109 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216112 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.546834 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216115 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216119 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216122 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216125 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216127 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216130 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216133 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216136 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216139 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216141 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216144 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216147 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216150 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216152 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216158 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216161 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216164 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216166 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216169 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216172 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.547371 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216174 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216177 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216180 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216182 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216185 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216188 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216208 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216212 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216214 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216218 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216220 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216223 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216227 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216230 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216234 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216237 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216240 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216243 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216245 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216248 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.547911 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216251 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216254 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216258 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216263 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216265 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216268 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216272 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216275 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216278 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216281 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216284 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216286 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216289 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216292 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216295 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216297 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216300 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216303 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216306 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.548459 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216308 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216311 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216314 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216317 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216320 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216323 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216327 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216331 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216335 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216338 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216341 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216345 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.216351 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216478 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 21:22:04.548969 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216483 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216487 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216490 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216493 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216496 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216501 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216504 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216507 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216509 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216512 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216515 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216518 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216520 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216523 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216526 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216530 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216532 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216535 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216538 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216541 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 21:22:04.549372 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216545 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216548 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216550 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216553 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216560 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216562 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216565 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216568 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216571 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216574 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216577 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216580 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216583 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216586 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216588 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216591 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216594 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216599 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216601 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216604 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 21:22:04.549909 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216607 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216610 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216612 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216615 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216618 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216620 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216623 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216626 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216629 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216632 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216634 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216637 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216640 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216642 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216645 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216648 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216653 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216657 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216660 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216663 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 21:22:04.550543 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216667 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216670 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216674 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216676 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216679 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216682 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216686 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216689 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216692 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216697 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216700 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216703 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216705 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216708 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216711 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216713 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216716 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216719 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216723 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 21:22:04.551122 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216726 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216729 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216732 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216734 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216737 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216740 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216742 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216745 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216747 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216752 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216755 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216757 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216760 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216763 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216771 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216774 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216784 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216789 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 21:22:04.551640 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:04.216792 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.216798 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.216914 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.220747 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.222263 2564 server.go:1019] "Starting client certificate rotation" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.224814 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.226809 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.226833 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.228214 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.240439 2564 log.go:25] "Validated CRI v1 runtime API" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.246465 2564 log.go:25] "Validated CRI v1 image API" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.247395 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 21:22:04.552112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.257031 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:22:04.552473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.261313 2564 fs.go:135] Filesystem UUIDs: map[5197f07e-cc8c-42ee-910c-6022218ff894:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eecd3988-4d5f-4f53-a2c6-5687aaebe9a4:/dev/nvme0n1p3] Jul 13 21:22:04.552473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.261329 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.265835 2564 manager.go:216] Machine: {Timestamp:2026-07-13 21:22:04.26530924 +0000 UTC m=+0.319843821 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100175 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23f5778ab7d5aa118a13414265c4a9 SystemUUID:ec23f577-8ab7-d5aa-118a-13414265c4a9 BootID:9434bfd0-db16-4a9f-ba64-699dd8a5dd54 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:1c:cc:5a:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:1c:cc:5a:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:4c:57:9b:3b:8b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.265942 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.266029 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.268636 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.268667 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-109.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.269445 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.269456 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.269469 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.270045 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.271426 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275418 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275538 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275572 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275582 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275597 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275609 2564 kubelet.go:398] "Adding apiserver pod source" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275618 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 21:22:04.552541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275981 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.275997 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.278049 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.280240 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.280290 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281024 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281046 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281054 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281061 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281074 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281083 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281096 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281105 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281116 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281125 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281136 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281143 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281817 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.281830 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.285043 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m66f2" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.285989 2564 server.go:1268] "Started kubelet" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.286508 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.288364 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.289428 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.289497 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.290652 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.291963 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m66f2" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.294658 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.297481 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.297767 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.297860 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.297876 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.297953 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298328 2564 factory.go:55] Registering systemd factory Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298345 2564 factory.go:223] Registration of the systemd container factory successfully Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298582 2564 factory.go:153] Registering CRI-O factory Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298596 2564 factory.go:223] Registration of the crio container factory successfully Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298669 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298688 2564 factory.go:103] Registering Raw factory Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298706 2564 manager.go:1196] Started watching for new ooms in manager Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298718 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.298730 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.313122 2564 manager.go:349] Starting recovery of all containers Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.316628 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-109.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.316653 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.316676 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 21:22:04.553062 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.316755 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.316819 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.316909 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.336157 2564 manager.go:354] Recovery completed Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.340991 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343115 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343140 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343152 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343796 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343804 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.343822 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.346350 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.347127 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.347158 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.347171 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.348224 2564 policy_none.go:49] "None policy: Start" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.348243 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.348256 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.350377 2564 policy_none.go:47] "Start" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.398397 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.409916 2564 manager.go:337] "Starting Device Plugin manager" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.409949 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.409959 2564 server.go:87] "Starting device plugin registration server" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.410312 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.410325 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.410439 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.410562 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.410573 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.411187 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.411235 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.511963 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.513241 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.513287 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.513304 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.513342 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.520863 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.522211 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.522255 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.522300 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.522337 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.530840 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.531568 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.554431 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.531596 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-109.ec2.internal\": node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.570543 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.570518 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.623804 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.623160 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal"] Jul 13 21:22:04.623898 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.623855 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.624903 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.624882 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.624996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.624923 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.624996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.624940 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.627595 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.627581 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.627749 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.627733 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.627792 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.627767 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.628374 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628352 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.628486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628385 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.628486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628397 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.628486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628358 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.628486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628464 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.628486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.628476 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.630855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.630838 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.630912 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.630869 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 21:22:04.631595 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.631572 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientMemory" Jul 13 21:22:04.631690 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.631609 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 21:22:04.631690 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.631625 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeHasSufficientPID" Jul 13 21:22:04.661051 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.661028 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-109.ec2.internal\" not found" node="ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.665877 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.665860 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-109.ec2.internal\" not found" node="ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.671020 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.671003 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.714275 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.714248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.714416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.714281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.714416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.714302 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3bea4fb0630134d8ede642fba0ebbd5f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-109.ec2.internal\" (UID: \"3bea4fb0630134d8ede642fba0ebbd5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.771674 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.771636 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.815213 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3bea4fb0630134d8ede642fba0ebbd5f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-109.ec2.internal\" (UID: \"3bea4fb0630134d8ede642fba0ebbd5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.815213 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.815213 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.815428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815237 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3bea4fb0630134d8ede642fba0ebbd5f-config\") pod \"kube-apiserver-proxy-ip-10-0-131-109.ec2.internal\" (UID: \"3bea4fb0630134d8ede642fba0ebbd5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.815428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.815428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.815281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48e3d4ce92082fbd206ffa91a6566245-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal\" (UID: \"48e3d4ce92082fbd206ffa91a6566245\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.875897 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.875848 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:04.966169 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.966139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.968802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:04.968781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:04.975978 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:04.975950 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.080280 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.080163 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.170611 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.170586 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:05.181084 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.181054 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.222194 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.222166 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 21:22:05.222361 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.222338 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 21:22:05.222420 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.222361 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 21:22:05.281220 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.281179 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.297831 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.297771 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:17:04 +0000 UTC" deadline="2028-03-09 10:28:54.526550403 +0000 UTC" Jul 13 21:22:05.297831 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.297826 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14509h6m49.228726388s" Jul 13 21:22:05.383982 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.383900 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.484120 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.484087 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.584762 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.584725 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.685362 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:05.685269 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-109.ec2.internal\" not found" Jul 13 21:22:05.724061 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:05.724026 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bea4fb0630134d8ede642fba0ebbd5f.slice/crio-f5fbfc79d7d20c568059bce024e41b699db9aea422f7ec22dc1c612fb544fe19 WatchSource:0}: Error finding container f5fbfc79d7d20c568059bce024e41b699db9aea422f7ec22dc1c612fb544fe19: Status 404 returned error can't find the container with id f5fbfc79d7d20c568059bce024e41b699db9aea422f7ec22dc1c612fb544fe19 Jul 13 21:22:05.724668 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:05.724642 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e3d4ce92082fbd206ffa91a6566245.slice/crio-10d06382d6e9be066148935c41480854a3c477f864dd87c0ca8fa555fe2932ae WatchSource:0}: Error finding container 10d06382d6e9be066148935c41480854a3c477f864dd87c0ca8fa555fe2932ae: Status 404 returned error can't find the container with id 10d06382d6e9be066148935c41480854a3c477f864dd87c0ca8fa555fe2932ae Jul 13 21:22:05.728882 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.728867 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:22:05.768385 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.768352 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:05.798028 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.797993 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" Jul 13 21:22:05.809207 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.809186 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:22:05.810957 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.810941 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" Jul 13 21:22:05.819038 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.819018 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 21:22:05.863823 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:05.863796 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:06.279229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.279194 2564 apiserver.go:52] "Watching apiserver" Jul 13 21:22:06.285470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.285444 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 21:22:06.286034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.285817 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-72bf7","kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal","openshift-cluster-node-tuning-operator/tuned-k6csn","openshift-dns/node-resolver-wrdfd","openshift-image-registry/node-ca-nt5rd","openshift-multus/multus-additional-cni-plugins-tlk2k","openshift-multus/multus-ljntd","openshift-multus/network-metrics-daemon-x4nrq","kube-system/global-pull-secret-syncer-ffpgk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal","openshift-network-diagnostics/network-check-target-whfmn","openshift-network-operator/iptables-alerter-5hgbf","openshift-ovn-kubernetes/ovnkube-node-kksxm"] Jul 13 21:22:06.290337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.290320 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.292559 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.292534 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.292686 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.292646 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.293113 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.293092 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.293261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.293150 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 21:22:06.293261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.293238 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 21:22:06.293343 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.293300 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xvhtb\"" Jul 13 21:22:06.295103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.294947 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4k7nl\"" Jul 13 21:22:06.295378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.295340 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.296019 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.295691 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 21:22:06.296019 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.295751 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.296912 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.296317 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.298495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.298462 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.298602 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.298513 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d87qc\"" Jul 13 21:22:06.298954 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.298928 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.299050 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.298986 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.299166 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.299151 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 21:22:06.301213 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.301194 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p2drv\"" Jul 13 21:22:06.301518 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.301503 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.301756 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.301738 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.302105 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.302086 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.303675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.303659 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.303896 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.303876 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 21:22:06.303996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.303885 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.304117 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.304095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.304542 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.304513 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zqmlc\"" Jul 13 21:22:06.305997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.305978 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7qwxx\"" Jul 13 21:22:06.306371 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.306349 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 21:22:06.306466 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.306377 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 21:22:06.306623 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.306599 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.306728 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.306676 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:06.309408 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.309034 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.311068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.311046 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 21:22:06.311344 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.311328 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 21:22:06.311428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.311364 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kc6n4\"" Jul 13 21:22:06.313577 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.313555 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.315562 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.315540 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 21:22:06.315672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.315602 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 21:22:06.316220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.316202 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.316357 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.316226 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 21:22:06.316452 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.316306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.316647 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.316343 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 21:22:06.317159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.317095 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.317159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.317109 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dmk62\"" Jul 13 21:22:06.317329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.317172 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 21:22:06.318599 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.318577 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 21:22:06.318698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.318645 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 21:22:06.318698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.318648 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 21:22:06.318817 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.318742 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pwlld\"" Jul 13 21:22:06.319106 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.319084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.319229 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.319172 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:06.321649 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.321629 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:06.321732 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.321690 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:06.329842 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-tmp\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.329959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77492200-648f-408d-8463-d57da27d2b17-hosts-file\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.329959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-os-release\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.329959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-modprobe-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330120 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-lib-modules\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330120 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.329986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-k8s-cni-cncf-io\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330120 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-multus\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330120 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-multus-certs\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330120 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktqv\" (UniqueName: \"kubernetes.io/projected/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-kube-api-access-cktqv\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-bin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330167 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-hostroot\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-conf-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-daemon-config\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330259 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8f9l\" (UniqueName: \"kubernetes.io/projected/2c34b003-0638-48db-8233-92915efaeb12-kube-api-access-m8f9l\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-kubelet\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330319 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hrh\" (UniqueName: \"kubernetes.io/projected/f910000b-1058-41e4-aee6-7ef8b4dda23c-kube-api-access-78hrh\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-kubernetes\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330370 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-system-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-cni-binary-copy\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330418 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-netns\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-etc-kubernetes\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-iptables-alerter-script\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-var-lib-kubelet\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330563 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-cnibin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-socket-dir-parent\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysconfig\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-conf\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-host\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-systemd\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.330884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.330756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-etc-tuned\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.331557 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77492200-648f-408d-8463-d57da27d2b17-tmp-dir\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.331557 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331550 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npqlj\" (UniqueName: \"kubernetes.io/projected/77492200-648f-408d-8463-d57da27d2b17-kube-api-access-npqlj\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.331669 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331583 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-host-slash\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.331669 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331639 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.331774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-run\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.331774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.331716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-sys\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.335341 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.335320 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g9q4" Jul 13 21:22:06.343862 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.343835 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g9q4" Jul 13 21:22:06.399675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.399646 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 21:22:06.432713 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77492200-648f-408d-8463-d57da27d2b17-hosts-file\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-os-release\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-kubernetes\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-system-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-cni-binary-copy\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.432908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432903 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-dbus\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-host\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.432977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77492200-648f-408d-8463-d57da27d2b17-tmp-dir\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46100359-9719-40a0-a391-72bae71f86ab-serviceca\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9z4\" (UniqueName: \"kubernetes.io/projected/ace0c1d9-d91d-40b4-b159-3996fbd78eff-kube-api-access-5l9z4\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433106 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npqlj\" (UniqueName: \"kubernetes.io/projected/77492200-648f-408d-8463-d57da27d2b17-kube-api-access-npqlj\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-netns\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433208 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.433276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/77492200-648f-408d-8463-d57da27d2b17-hosts-file\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-socket-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433324 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-host-slash\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-run\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433431 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-sys\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-kubelet\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433483 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/803bda74-adc9-4f4a-937f-d41286d252b8-kube-api-access-pdxqd\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-kubernetes\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-os-release\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-modprobe-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77492200-648f-408d-8463-d57da27d2b17-tmp-dir\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt68s\" (UniqueName: \"kubernetes.io/projected/46100359-9719-40a0-a391-72bae71f86ab-kube-api-access-qt68s\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433683 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-host\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-config\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.433810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433729 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-run\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433762 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7gs\" (UniqueName: \"kubernetes.io/projected/29e5c617-246e-46c2-9841-407b07155568-kube-api-access-cv7gs\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-sys\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-conf-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433823 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-host-slash\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-cni-binary-copy\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-ovn\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433887 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-os-release\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-system-cni-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-node-log\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-kubelet\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.433994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-conf-dir\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-modprobe-d\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-kubelet-config\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.434709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434092 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-kubelet\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-etc-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-log-socket\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-bin\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434251 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5c617-246e-46c2-9841-407b07155568-ovn-node-metrics-cert\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c1e3bd8a-2beb-44a1-88af-c230ed453992-agent-certs\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-netns\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434381 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-etc-kubernetes\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434415 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-etc-kubernetes\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434496 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-netns\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-iptables-alerter-script\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-var-lib-kubelet\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-slash\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-var-lib-kubelet\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-cnibin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-socket-dir-parent\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysconfig\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.435591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-conf\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434819 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-etc-tuned\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434878 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgl5b\" (UniqueName: \"kubernetes.io/projected/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-kube-api-access-kgl5b\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-systemd\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.434994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cnibin\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435020 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-tmp\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435025 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-iptables-alerter-script\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-sys-fs\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435097 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-lib-modules\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c1e3bd8a-2beb-44a1-88af-c230ed453992-konnectivity-ca\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435171 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysctl-conf\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435224 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435253 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-systemd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435278 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-device-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.436546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-lib-modules\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-k8s-cni-cncf-io\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-multus\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435361 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-k8s-cni-cncf-io\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-multus-certs\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cktqv\" (UniqueName: \"kubernetes.io/projected/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-kube-api-access-cktqv\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435414 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-sysconfig\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46100359-9719-40a0-a391-72bae71f86ab-host\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435465 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435494 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-cnibin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-netd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c34b003-0638-48db-8233-92915efaeb12-etc-systemd\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-registration-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435496 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-socket-dir-parent\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435651 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-bin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-hostroot\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-daemon-config\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.437472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435732 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8f9l\" (UniqueName: \"kubernetes.io/projected/2c34b003-0638-48db-8233-92915efaeb12-kube-api-access-m8f9l\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435756 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-run-multus-certs\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435820 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-system-cni-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-bin\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-systemd-units\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435889 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-host-var-lib-cni-multus\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-var-lib-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-env-overrides\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.435974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hrh\" (UniqueName: \"kubernetes.io/projected/f910000b-1058-41e4-aee6-7ef8b4dda23c-kube-api-access-78hrh\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.436025 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-script-lib\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.436073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f910000b-1058-41e4-aee6-7ef8b4dda23c-hostroot\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.436082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:06.438396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.436448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f910000b-1058-41e4-aee6-7ef8b4dda23c-multus-daemon-config\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.439603 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.439579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-etc-tuned\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.441760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.441711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c34b003-0638-48db-8233-92915efaeb12-tmp\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.450346 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.450269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npqlj\" (UniqueName: \"kubernetes.io/projected/77492200-648f-408d-8463-d57da27d2b17-kube-api-access-npqlj\") pod \"node-resolver-wrdfd\" (UID: \"77492200-648f-408d-8463-d57da27d2b17\") " pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.450654 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.450629 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktqv\" (UniqueName: \"kubernetes.io/projected/4ec6e00e-4da2-4a92-ac77-206fdc9e8f08-kube-api-access-cktqv\") pod \"iptables-alerter-5hgbf\" (UID: \"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08\") " pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.450739 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.450662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8f9l\" (UniqueName: \"kubernetes.io/projected/2c34b003-0638-48db-8233-92915efaeb12-kube-api-access-m8f9l\") pod \"tuned-k6csn\" (UID: \"2c34b003-0638-48db-8233-92915efaeb12\") " pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.452865 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.452769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hrh\" (UniqueName: \"kubernetes.io/projected/f910000b-1058-41e4-aee6-7ef8b4dda23c-kube-api-access-78hrh\") pod \"multus-ljntd\" (UID: \"f910000b-1058-41e4-aee6-7ef8b4dda23c\") " pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.528872 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.528792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" event={"ID":"48e3d4ce92082fbd206ffa91a6566245","Type":"ContainerStarted","Data":"10d06382d6e9be066148935c41480854a3c477f864dd87c0ca8fa555fe2932ae"} Jul 13 21:22:06.530723 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.530645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" event={"ID":"3bea4fb0630134d8ede642fba0ebbd5f","Type":"ContainerStarted","Data":"f5fbfc79d7d20c568059bce024e41b699db9aea422f7ec22dc1c612fb544fe19"} Jul 13 21:22:06.537282 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-kubelet-config\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.537409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-etc-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.537409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-log-socket\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.537409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-bin\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.537409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5c617-246e-46c2-9841-407b07155568-ovn-node-metrics-cert\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.537409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537399 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c1e3bd8a-2beb-44a1-88af-c230ed453992-agent-certs\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-slash\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgl5b\" (UniqueName: \"kubernetes.io/projected/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-kube-api-access-kgl5b\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cnibin\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-sys-fs\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c1e3bd8a-2beb-44a1-88af-c230ed453992-konnectivity-ca\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.537675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-systemd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-device-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46100359-9719-40a0-a391-72bae71f86ab-host\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-netd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-registration-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-system-cni-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-systemd-units\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-var-lib-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537882 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-env-overrides\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-script-lib\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537934 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.537985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-os-release\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-dbus\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46100359-9719-40a0-a391-72bae71f86ab-serviceca\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9z4\" (UniqueName: \"kubernetes.io/projected/ace0c1d9-d91d-40b4-b159-3996fbd78eff-kube-api-access-5l9z4\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.538124 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-netns\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-socket-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538258 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-kubelet\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538284 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/803bda74-adc9-4f4a-937f-d41286d252b8-kube-api-access-pdxqd\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt68s\" (UniqueName: \"kubernetes.io/projected/46100359-9719-40a0-a391-72bae71f86ab-kube-api-access-qt68s\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-config\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7gs\" (UniqueName: \"kubernetes.io/projected/29e5c617-246e-46c2-9841-407b07155568-kube-api-access-cv7gs\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538447 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-ovn\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-node-log\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-node-log\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538675 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-kubelet-config\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.538959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-etc-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.539921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538752 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-log-socket\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.539921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.538789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-bin\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.539921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.539280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-systemd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.539921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.539692 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cnibin\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.539921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.539884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540226 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540026 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540226 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540157 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-socket-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.540336 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.540229 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:06.540336 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.540306 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:07.040281676 +0000 UTC m=+3.094816265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:06.540336 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.540517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540342 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-slash\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540384 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-os-release\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.540517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-sys-fs\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.540517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540480 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-script-lib\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540524 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5209694d-9769-4c3f-be50-c900e8628640-dbus\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540530 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-device-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46100359-9719-40a0-a391-72bae71f86ab-host\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-cni-netd\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540675 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-kubelet\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.540747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-host-run-netns\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.540784 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-system-cni-dir\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46100359-9719-40a0-a391-72bae71f86ab-serviceca\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541072 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-ovnkube-config\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-systemd-units\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-etc-selinux\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/803bda74-adc9-4f4a-937f-d41286d252b8-registration-dir\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-var-lib-openvswitch\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541290 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.541375 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.541421 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:07.041404977 +0000 UTC m=+3.095939560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541467 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29e5c617-246e-46c2-9841-407b07155568-run-ovn\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29e5c617-246e-46c2-9841-407b07155568-env-overrides\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.541895 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541740 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c1e3bd8a-2beb-44a1-88af-c230ed453992-konnectivity-ca\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.541895 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.541739 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.542318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.542296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c1e3bd8a-2beb-44a1-88af-c230ed453992-agent-certs\") pod \"konnectivity-agent-72bf7\" (UID: \"c1e3bd8a-2beb-44a1-88af-c230ed453992\") " pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.543609 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.543585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5c617-246e-46c2-9841-407b07155568-ovn-node-metrics-cert\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.545630 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.545611 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:06.545746 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.545638 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:06.545746 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.545653 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:06.545746 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:06.545715 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:07.045698528 +0000 UTC m=+3.100233121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:06.552174 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.552125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt68s\" (UniqueName: \"kubernetes.io/projected/46100359-9719-40a0-a391-72bae71f86ab-kube-api-access-qt68s\") pod \"node-ca-nt5rd\" (UID: \"46100359-9719-40a0-a391-72bae71f86ab\") " pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.552898 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.552850 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgl5b\" (UniqueName: \"kubernetes.io/projected/d66c5288-d9bf-4cb3-9321-7308fcf01ae3-kube-api-access-kgl5b\") pod \"multus-additional-cni-plugins-tlk2k\" (UID: \"d66c5288-d9bf-4cb3-9321-7308fcf01ae3\") " pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.554322 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.554277 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxqd\" (UniqueName: \"kubernetes.io/projected/803bda74-adc9-4f4a-937f-d41286d252b8-kube-api-access-pdxqd\") pod \"aws-ebs-csi-driver-node-qtbz7\" (UID: \"803bda74-adc9-4f4a-937f-d41286d252b8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.554897 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.554857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7gs\" (UniqueName: \"kubernetes.io/projected/29e5c617-246e-46c2-9841-407b07155568-kube-api-access-cv7gs\") pod \"ovnkube-node-kksxm\" (UID: \"29e5c617-246e-46c2-9841-407b07155568\") " pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.555794 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.555751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9z4\" (UniqueName: \"kubernetes.io/projected/ace0c1d9-d91d-40b4-b159-3996fbd78eff-kube-api-access-5l9z4\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:06.576386 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.576360 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:06.605616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.605583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ljntd" Jul 13 21:22:06.612706 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.612681 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hgbf" Jul 13 21:22:06.622799 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.622605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6csn" Jul 13 21:22:06.624707 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.624673 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf910000b_1058_41e4_aee6_7ef8b4dda23c.slice/crio-1006b5b5aaa6530a6230f83cb32f8a10922c495425d29795ec44ee7ffcba550d WatchSource:0}: Error finding container 1006b5b5aaa6530a6230f83cb32f8a10922c495425d29795ec44ee7ffcba550d: Status 404 returned error can't find the container with id 1006b5b5aaa6530a6230f83cb32f8a10922c495425d29795ec44ee7ffcba550d Jul 13 21:22:06.627173 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.627150 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec6e00e_4da2_4a92_ac77_206fdc9e8f08.slice/crio-1173f1d6adfc8c7dcbecb3dc885df2a7fc785bdf16727dd6b6e81cce2a00a595 WatchSource:0}: Error finding container 1173f1d6adfc8c7dcbecb3dc885df2a7fc785bdf16727dd6b6e81cce2a00a595: Status 404 returned error can't find the container with id 1173f1d6adfc8c7dcbecb3dc885df2a7fc785bdf16727dd6b6e81cce2a00a595 Jul 13 21:22:06.630415 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.630390 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:06.637473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.637265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wrdfd" Jul 13 21:22:06.638975 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.638951 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c34b003_0638_48db_8233_92915efaeb12.slice/crio-36a2acb546b24a82cd730179c7b3a2136b0fa1a745eceb5c8b5636d7f9e7134a WatchSource:0}: Error finding container 36a2acb546b24a82cd730179c7b3a2136b0fa1a745eceb5c8b5636d7f9e7134a: Status 404 returned error can't find the container with id 36a2acb546b24a82cd730179c7b3a2136b0fa1a745eceb5c8b5636d7f9e7134a Jul 13 21:22:06.642705 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.642677 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e5c617_246e_46c2_9841_407b07155568.slice/crio-41ec3f7b5e08b7c1d5003e31025fe7dcec90b38764afb6a4c66019e513120fa6 WatchSource:0}: Error finding container 41ec3f7b5e08b7c1d5003e31025fe7dcec90b38764afb6a4c66019e513120fa6: Status 404 returned error can't find the container with id 41ec3f7b5e08b7c1d5003e31025fe7dcec90b38764afb6a4c66019e513120fa6 Jul 13 21:22:06.645402 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.645383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nt5rd" Jul 13 21:22:06.649784 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.649759 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77492200_648f_408d_8463_d57da27d2b17.slice/crio-31eee03fcfb02985f857391b095d757e23da86089e46cc9bb632db2fc7de9af2 WatchSource:0}: Error finding container 31eee03fcfb02985f857391b095d757e23da86089e46cc9bb632db2fc7de9af2: Status 404 returned error can't find the container with id 31eee03fcfb02985f857391b095d757e23da86089e46cc9bb632db2fc7de9af2 Jul 13 21:22:06.653404 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.653381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" Jul 13 21:22:06.658267 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.658232 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46100359_9719_40a0_a391_72bae71f86ab.slice/crio-52e24de2e7f1da951123df28bce849f61c93ef32903488ebacad3d6df7b65f93 WatchSource:0}: Error finding container 52e24de2e7f1da951123df28bce849f61c93ef32903488ebacad3d6df7b65f93: Status 404 returned error can't find the container with id 52e24de2e7f1da951123df28bce849f61c93ef32903488ebacad3d6df7b65f93 Jul 13 21:22:06.661933 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.661907 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:06.664296 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.664168 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66c5288_d9bf_4cb3_9321_7308fcf01ae3.slice/crio-19846774ee9a1647aa3d619215ea0f72c496373472711128c90f58517a872ce7 WatchSource:0}: Error finding container 19846774ee9a1647aa3d619215ea0f72c496373472711128c90f58517a872ce7: Status 404 returned error can't find the container with id 19846774ee9a1647aa3d619215ea0f72c496373472711128c90f58517a872ce7 Jul 13 21:22:06.666850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:06.666828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" Jul 13 21:22:06.674462 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.674429 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e3bd8a_2beb_44a1_88af_c230ed453992.slice/crio-a3dda4e8d78879ad7490e380bbc2b5a2c075e4bb1207b96c7221efa43ffdd35b WatchSource:0}: Error finding container a3dda4e8d78879ad7490e380bbc2b5a2c075e4bb1207b96c7221efa43ffdd35b: Status 404 returned error can't find the container with id a3dda4e8d78879ad7490e380bbc2b5a2c075e4bb1207b96c7221efa43ffdd35b Jul 13 21:22:06.678745 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:06.678719 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803bda74_adc9_4f4a_937f_d41286d252b8.slice/crio-dbcef208551026ee9c7f4f324867af560d449daf6c5da335af4d1304036c1504 WatchSource:0}: Error finding container dbcef208551026ee9c7f4f324867af560d449daf6c5da335af4d1304036c1504: Status 404 returned error can't find the container with id dbcef208551026ee9c7f4f324867af560d449daf6c5da335af4d1304036c1504 Jul 13 21:22:07.041535 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.041495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:07.041735 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.041586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:07.041735 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.041694 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:07.041735 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.041708 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:07.041882 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.041775 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:08.041760689 +0000 UTC m=+4.096295256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:07.041882 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.041796 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:08.041786413 +0000 UTC m=+4.096320995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:07.143043 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.143003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:07.143271 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.143224 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:07.143271 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.143252 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:07.143271 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.143268 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:07.143493 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.143335 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:08.143312256 +0000 UTC m=+4.197846841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:07.347476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.347357 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:17:06 +0000 UTC" deadline="2028-01-20 00:40:20.601284299 +0000 UTC" Jul 13 21:22:07.347476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.347426 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13323h18m13.25386369s" Jul 13 21:22:07.523367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.523330 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:07.523549 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.523486 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:07.524004 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.523957 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:07.524104 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:07.524080 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:07.535972 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.535881 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6csn" event={"ID":"2c34b003-0638-48db-8233-92915efaeb12","Type":"ContainerStarted","Data":"36a2acb546b24a82cd730179c7b3a2136b0fa1a745eceb5c8b5636d7f9e7134a"} Jul 13 21:22:07.538420 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.538339 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljntd" event={"ID":"f910000b-1058-41e4-aee6-7ef8b4dda23c","Type":"ContainerStarted","Data":"1006b5b5aaa6530a6230f83cb32f8a10922c495425d29795ec44ee7ffcba550d"} Jul 13 21:22:07.541441 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.541386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nt5rd" event={"ID":"46100359-9719-40a0-a391-72bae71f86ab","Type":"ContainerStarted","Data":"52e24de2e7f1da951123df28bce849f61c93ef32903488ebacad3d6df7b65f93"} Jul 13 21:22:07.544077 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.543969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"41ec3f7b5e08b7c1d5003e31025fe7dcec90b38764afb6a4c66019e513120fa6"} Jul 13 21:22:07.546195 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.546105 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hgbf" event={"ID":"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08","Type":"ContainerStarted","Data":"1173f1d6adfc8c7dcbecb3dc885df2a7fc785bdf16727dd6b6e81cce2a00a595"} Jul 13 21:22:07.550320 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.550179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" event={"ID":"803bda74-adc9-4f4a-937f-d41286d252b8","Type":"ContainerStarted","Data":"dbcef208551026ee9c7f4f324867af560d449daf6c5da335af4d1304036c1504"} Jul 13 21:22:07.558293 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.558241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72bf7" event={"ID":"c1e3bd8a-2beb-44a1-88af-c230ed453992","Type":"ContainerStarted","Data":"a3dda4e8d78879ad7490e380bbc2b5a2c075e4bb1207b96c7221efa43ffdd35b"} Jul 13 21:22:07.559980 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.559952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerStarted","Data":"19846774ee9a1647aa3d619215ea0f72c496373472711128c90f58517a872ce7"} Jul 13 21:22:07.562960 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:07.562903 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wrdfd" event={"ID":"77492200-648f-408d-8463-d57da27d2b17","Type":"ContainerStarted","Data":"31eee03fcfb02985f857391b095d757e23da86089e46cc9bb632db2fc7de9af2"} Jul 13 21:22:08.051156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.051113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:08.051365 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.051218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:08.051365 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.051286 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:08.051486 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.051365 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:08.051486 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.051373 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:10.0513518 +0000 UTC m=+6.105886370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:08.051486 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.051434 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:10.05141704 +0000 UTC m=+6.105951614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:08.091482 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.091447 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 21:22:08.156313 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.156268 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:08.156313 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.156300 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:08.156313 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.156316 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:08.156629 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.156378 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:10.156357904 +0000 UTC m=+6.210892474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:08.156941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.156831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:08.347676 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.347605 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:17:06 +0000 UTC" deadline="2028-02-25 00:42:57.432775681 +0000 UTC" Jul 13 21:22:08.347676 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.347676 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14187h20m49.085104565s" Jul 13 21:22:08.524637 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:08.524041 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:08.524637 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:08.524218 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:09.524209 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:09.523437 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:09.524209 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:09.523593 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:09.524209 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:09.524037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:09.524209 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:09.524139 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:10.074324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:10.074410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.074589 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.074662 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:14.074642941 +0000 UTC m=+10.129177530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.075111 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:10.075231 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.075173 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:14.075154036 +0000 UTC m=+10.129688612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:10.175894 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:10.175851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:10.176110 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.176042 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:10.176110 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.176067 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:10.176110 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.176082 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:10.176315 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.176161 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:14.176123813 +0000 UTC m=+10.230658395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:10.525610 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:10.525519 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:10.526085 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:10.525671 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:11.523834 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:11.523189 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:11.523834 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:11.523330 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:11.523834 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:11.523704 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:11.523834 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:11.523791 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:12.523360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:12.523054 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:12.523807 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:12.523448 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:13.523490 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:13.523445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:13.523944 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:13.523576 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:13.523944 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:13.523452 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:13.524068 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:13.524019 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:14.111077 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:14.111039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:14.111248 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:14.111148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:14.111248 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.111218 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:14.111369 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.111261 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:14.111369 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.111322 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:22.111299617 +0000 UTC m=+18.165834185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:14.111369 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.111345 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:22.111332111 +0000 UTC m=+18.165866683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:14.212358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:14.212319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:14.212537 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.212496 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:14.212537 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.212514 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:14.212537 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.212528 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:14.212676 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.212588 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:22.21256864 +0000 UTC m=+18.267103210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:14.523760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:14.523671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:14.524232 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:14.523790 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:15.523180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:15.522605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:15.523180 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:15.522752 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:15.523180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:15.522770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:15.523180 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:15.522842 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:16.523596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:16.523394 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:16.523596 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:16.523528 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:17.522770 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:17.522732 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:17.522934 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:17.522745 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:17.522934 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:17.522857 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:17.523007 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:17.522960 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:18.523044 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:18.523001 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:18.523466 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:18.523141 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:19.522952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:19.522917 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:19.523163 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:19.522954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:19.523163 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:19.523040 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:19.523561 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:19.523176 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:20.522984 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:20.522951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:20.523147 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:20.523084 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:21.523127 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:21.523079 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:21.523522 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:21.523205 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:21.523522 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:21.523264 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:21.523522 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:21.523355 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:22.173838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:22.173802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:22.174045 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:22.173873 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:22.174045 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.173938 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:22.174045 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.174004 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:22:38.173987005 +0000 UTC m=+34.228521583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:22.174241 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.174043 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:22.174241 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.174107 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:38.174092317 +0000 UTC m=+34.228626882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:22.274935 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:22.274883 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:22.275136 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.275104 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:22.275199 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.275150 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:22.275199 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.275165 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:22.275294 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.275236 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:22:38.275215921 +0000 UTC m=+34.329750507 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:22.523256 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:22.523172 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:22.523690 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:22.523311 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:23.523530 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:23.523494 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:23.523971 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:23.523502 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:23.523971 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:23.523615 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:23.523971 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:23.523690 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:24.523954 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:24.523916 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:24.524382 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:24.524032 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:25.523485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:25.523445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:25.523676 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:25.523495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:25.523676 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:25.523613 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:25.523782 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:25.523705 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:26.524686 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.523724 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:26.524686 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:26.523853 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:26.605270 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.605231 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6csn" event={"ID":"2c34b003-0638-48db-8233-92915efaeb12","Type":"ContainerStarted","Data":"1e76db7fc4565ee7376d1646bf74fe8168835a21590c182a35c5f6d7f506e0ea"} Jul 13 21:22:26.611917 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.611888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ljntd" event={"ID":"f910000b-1058-41e4-aee6-7ef8b4dda23c","Type":"ContainerStarted","Data":"8183784e9fe8fa20b6caf491b3b8765432e8d6c1500c80b51c5a5dbb42e5e6da"} Jul 13 21:22:26.616450 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.616386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" event={"ID":"3bea4fb0630134d8ede642fba0ebbd5f","Type":"ContainerStarted","Data":"6fb2a09e53b3eeff39c340d505f0d75aee0892c7ea55f47439e821fc9c2f7344"} Jul 13 21:22:26.624829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.624782 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k6csn" podStartSLOduration=2.014783467 podStartE2EDuration="21.624763866s" totalImagesPullingTime="19.609980399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:26.624563017 +0000 UTC m=+22.679097605" watchObservedRunningTime="2026-07-13 21:22:26.624763866 +0000 UTC m=+22.679298450" Jul 13 21:22:26.636947 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.636739 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-109.ec2.internal" podStartSLOduration=20.636719465 podStartE2EDuration="20.636719465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:26.636666953 +0000 UTC m=+22.691201540" watchObservedRunningTime="2026-07-13 21:22:26.636719465 +0000 UTC m=+22.691254052" Jul 13 21:22:26.655853 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:26.655757 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ljntd" podStartSLOduration=2.012287297 podStartE2EDuration="21.655732378s" totalImagesPullingTime="19.643445081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:26.654969263 +0000 UTC m=+22.709503852" watchObservedRunningTime="2026-07-13 21:22:26.655732378 +0000 UTC m=+22.710266966" Jul 13 21:22:27.522981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.522800 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:27.523145 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.522825 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:27.523145 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:27.523070 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:27.523145 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:27.523109 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:27.620807 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.620763 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" event={"ID":"803bda74-adc9-4f4a-937f-d41286d252b8","Type":"ContainerStarted","Data":"d00124cf36d7b856dc900ffd287b6a5c21d32774b48b81ed74b3e946f4e6f362"} Jul 13 21:22:27.622434 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.622399 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72bf7" event={"ID":"c1e3bd8a-2beb-44a1-88af-c230ed453992","Type":"ContainerStarted","Data":"8de827f0bab4cc2263c70125eea28b0a3a2ab78dd257c099c532d8d42d033b63"} Jul 13 21:22:27.624021 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.623996 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="cd1a16525b715ab3a32d2dd420b84de3fd7a320b86792e7c02acf45657cdfc43" exitCode=0 Jul 13 21:22:27.624148 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.624075 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"cd1a16525b715ab3a32d2dd420b84de3fd7a320b86792e7c02acf45657cdfc43"} Jul 13 21:22:27.625856 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.625670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wrdfd" event={"ID":"77492200-648f-408d-8463-d57da27d2b17","Type":"ContainerStarted","Data":"9ac397a5e6e79711071ce0f2c0a15501912e80999a74cd60cd77f782b823b2f6"} Jul 13 21:22:27.627297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.627266 2564 generic.go:356] "Generic (PLEG): container finished" podID="48e3d4ce92082fbd206ffa91a6566245" containerID="6761a0db9e8652976cf7d9186e1b71c32fe211398d66ea61e67ff8996ddedb62" exitCode=0 Jul 13 21:22:27.627395 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.627353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" event={"ID":"48e3d4ce92082fbd206ffa91a6566245","Type":"ContainerDied","Data":"6761a0db9e8652976cf7d9186e1b71c32fe211398d66ea61e67ff8996ddedb62"} Jul 13 21:22:27.628875 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.628852 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nt5rd" event={"ID":"46100359-9719-40a0-a391-72bae71f86ab","Type":"ContainerStarted","Data":"f6fec694aacbb2d297be2d5973580772de7b77eea0628549c64693644b3acb3e"} Jul 13 21:22:27.632103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632078 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"39065d79f83becd106606a726ed5be1a62732ab447d4755639ca2c131a18968d"} Jul 13 21:22:27.632207 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632156 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"0782fd1ef4a5311e040906567727fe9b3797d12e6b6f5b963673b90a464423e9"} Jul 13 21:22:27.632207 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632192 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"3ff45c03942c573e975e40f28cf824acd7be98aec0a5b3b33dd708362842a20b"} Jul 13 21:22:27.632300 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"a5085c7363b6ae9dd0d65ea7268d4b53e6146a9f7dd9cf52ae245fca02b5e113"} Jul 13 21:22:27.632300 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632267 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"4597a2178a335428f2c5de38c9509d1136b4b522cb1af104ef485fd8920ee105"} Jul 13 21:22:27.632359 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.632315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"e0a3c9e7d79dc6a0abb5cb6dbdd6050d00b24ebddde8c492236546f3c2170674"} Jul 13 21:22:27.636132 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.636075 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-72bf7" podStartSLOduration=11.525337291 podStartE2EDuration="22.636060243s" totalImagesPullingTime="11.110722952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:27.635667007 +0000 UTC m=+23.690201597" watchObservedRunningTime="2026-07-13 21:22:27.636060243 +0000 UTC m=+23.690594830" Jul 13 21:22:27.648707 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.648662 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wrdfd" podStartSLOduration=3.049263813 podStartE2EDuration="22.648648962s" totalImagesPullingTime="19.599385149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:27.648399994 +0000 UTC m=+23.702934580" watchObservedRunningTime="2026-07-13 21:22:27.648648962 +0000 UTC m=+23.703183550" Jul 13 21:22:27.666270 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.666221 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nt5rd" podStartSLOduration=3.076106652 podStartE2EDuration="22.666201757s" totalImagesPullingTime="19.590095105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:27.665507067 +0000 UTC m=+23.720041654" watchObservedRunningTime="2026-07-13 21:22:27.666201757 +0000 UTC m=+23.720736345" Jul 13 21:22:27.768699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:27.768677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wrdfd_77492200-648f-408d-8463-d57da27d2b17/dns-node-resolver/0.log" Jul 13 21:22:28.080522 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.080441 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 21:22:28.417825 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.417749 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:28.423664 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.423576 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:22:28.080458217Z","UUID":"af86e1e1-4b30-4b14-8b93-9588ff275ba8","Handler":null,"Name":"","Endpoint":""} Jul 13 21:22:28.425191 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.425171 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 21:22:28.425311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.425254 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 21:22:28.523462 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.523432 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:28.523644 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:28.523568 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:28.546199 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.546174 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nt5rd_46100359-9719-40a0-a391-72bae71f86ab/node-ca/0.log" Jul 13 21:22:28.637087 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.637055 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" event={"ID":"48e3d4ce92082fbd206ffa91a6566245","Type":"ContainerStarted","Data":"50bd8e54dc1490042136a09bd1ac3bfefa7fc431919056238710614759a5ed2e"} Jul 13 21:22:28.639319 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.639024 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hgbf" event={"ID":"4ec6e00e-4da2-4a92-ac77-206fdc9e8f08","Type":"ContainerStarted","Data":"4f64a3a84f980647e1ba6a554147d0bc42385f1f3aef582e6e400e3cc46c3c37"} Jul 13 21:22:28.641503 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.641476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" event={"ID":"803bda74-adc9-4f4a-937f-d41286d252b8","Type":"ContainerStarted","Data":"44e17be8dc4b2d968555390f79631764cad6c831426ef1d4dc52cc6b85536c16"} Jul 13 21:22:28.651737 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.651683 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-109.ec2.internal" podStartSLOduration=22.651663585 podStartE2EDuration="22.651663585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:28.65060404 +0000 UTC m=+24.705138627" watchObservedRunningTime="2026-07-13 21:22:28.651663585 +0000 UTC m=+24.706198175" Jul 13 21:22:28.664638 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.664581 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5hgbf" podStartSLOduration=4.046641863 podStartE2EDuration="23.664562298s" totalImagesPullingTime="19.617920435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:28.663901278 +0000 UTC m=+24.718435866" watchObservedRunningTime="2026-07-13 21:22:28.664562298 +0000 UTC m=+24.719096886" Jul 13 21:22:28.784967 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.784889 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:28.873483 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.873450 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:28.933467 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:28.933438 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:29.026662 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.026619 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:29.027220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.027196 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:29.460290 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.460207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:29.522834 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.522799 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:29.523000 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.522895 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:29.523000 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:29.522958 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:29.523000 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:29.522983 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:29.646641 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.646564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"840d8c58cf29818fe92d65d642080dab6a7a5bdf2786b18c133b4bb10d4909c7"} Jul 13 21:22:29.649172 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.649144 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" event={"ID":"803bda74-adc9-4f4a-937f-d41286d252b8","Type":"ContainerStarted","Data":"b72dd1ca75a2970e42b9f23ff5cabe1c1f5c55c593d3701f40af693902aaa192"} Jul 13 21:22:29.649374 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.649355 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:29.650084 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.650055 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-72bf7" Jul 13 21:22:29.678324 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.678270 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qtbz7" podStartSLOduration=2.832903805 podStartE2EDuration="24.678249127s" totalImagesPullingTime="21.845345322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:29.668945143 +0000 UTC m=+25.723479728" watchObservedRunningTime="2026-07-13 21:22:29.678249127 +0000 UTC m=+25.732784124" Jul 13 21:22:29.754748 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.754711 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:29.945002 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:29.944924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:30.523055 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:30.523022 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:30.523402 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:30.523163 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:31.127899 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.127862 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:31.228922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.228857 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:31.523527 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.523496 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:31.523698 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:31.523624 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:31.523698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.523675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:31.523791 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:31.523768 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:31.615975 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.615944 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:31.787229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.787143 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:31.900513 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.900485 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:31.985653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:31.985621 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:32.523999 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.523462 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:32.523999 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:32.523742 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:32.552598 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.552567 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:32.660338 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.660265 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" event={"ID":"29e5c617-246e-46c2-9841-407b07155568","Type":"ContainerStarted","Data":"97c19dc6baf92adcc054d8f6021868a672883c13d4513b788308c0cecd6c9e44"} Jul 13 21:22:32.660560 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.660542 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:32.660636 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.660620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:32.660693 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.660684 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:32.675933 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.675910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:32.676052 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.676003 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:22:32.686531 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.686482 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" podStartSLOduration=7.821644722 podStartE2EDuration="27.686466816s" totalImagesPullingTime="19.864822094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:32.684947974 +0000 UTC m=+28.739482561" watchObservedRunningTime="2026-07-13 21:22:32.686466816 +0000 UTC m=+28.741001400" Jul 13 21:22:32.911734 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:32.911651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:33.315449 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.315344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:33.513064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.513036 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:33.523067 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.523049 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:33.523154 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.523048 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:33.523197 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:33.523171 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:33.523236 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:33.523202 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:33.664296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.664217 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="9a6a8c57ab3a665726aba4aaa2d478b3d71e22abf737c7bbcac9e4a82ef1049b" exitCode=0 Jul 13 21:22:33.665022 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.664304 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"9a6a8c57ab3a665726aba4aaa2d478b3d71e22abf737c7bbcac9e4a82ef1049b"} Jul 13 21:22:33.707951 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:33.707920 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:34.108865 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:34.108776 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:34.524370 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:34.524296 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:34.524531 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:34.524405 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:34.711408 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:34.711374 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:35.222181 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.222078 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:35.322914 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.322872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:35.423060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.423025 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:35.523033 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.522948 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:35.523227 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.522950 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:35.523227 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:35.523044 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:35.523227 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:35.523145 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:35.523940 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.523924 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:35.670595 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.670556 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="3c9ab5198046388ed0cb976cbed1950a87e40fc2b8b9173f35a46d3e19a98e27" exitCode=0 Jul 13 21:22:35.670820 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.670627 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"3c9ab5198046388ed0cb976cbed1950a87e40fc2b8b9173f35a46d3e19a98e27"} Jul 13 21:22:35.914841 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:35.914761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:36.103851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:36.103812 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:36.164631 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:36.164600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:36.523364 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:36.523332 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:36.523516 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:36.523467 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:36.705870 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:36.705837 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:36.909098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:36.909064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:37.510272 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.510239 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:37.523310 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.523275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:37.523310 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.523303 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:37.523510 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:37.523399 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:37.523563 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:37.523513 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:37.678193 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.678160 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="5d9d81670845255c558ab120285d56b7f44d176d21dc4d45a83a45ca66e671d6" exitCode=0 Jul 13 21:22:37.678360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.678209 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"5d9d81670845255c558ab120285d56b7f44d176d21dc4d45a83a45ca66e671d6"} Jul 13 21:22:37.710485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.710443 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:37.928241 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:37.928159 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:38.108616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.108589 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:38.189800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.189725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:38.189800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.189789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:38.190037 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.189881 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:38.190037 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.189946 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret podName:5209694d-9769-4c3f-be50-c900e8628640 nodeName:}" failed. No retries permitted until 2026-07-13 21:23:10.189928623 +0000 UTC m=+66.244463187 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret") pod "global-pull-secret-syncer-ffpgk" (UID: "5209694d-9769-4c3f-be50-c900e8628640") : object "kube-system"/"original-pull-secret" not registered Jul 13 21:22:38.190037 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.189887 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:38.190037 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.190026 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs podName:ace0c1d9-d91d-40b4-b159-3996fbd78eff nodeName:}" failed. No retries permitted until 2026-07-13 21:23:10.190008894 +0000 UTC m=+66.244543482 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs") pod "network-metrics-daemon-x4nrq" (UID: "ace0c1d9-d91d-40b4-b159-3996fbd78eff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 21:22:38.290607 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.290572 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:38.290748 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.290706 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 21:22:38.290748 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.290720 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 21:22:38.290748 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.290728 2564 projected.go:196] Error preparing data for projected volume kube-api-access-pfj96 for pod openshift-network-diagnostics/network-check-target-whfmn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:38.290873 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.290776 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96 podName:8db714f3-333b-4b45-8e33-9a6b9c1c00e9 nodeName:}" failed. No retries permitted until 2026-07-13 21:23:10.29076361 +0000 UTC m=+66.345298174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfj96" (UniqueName: "kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96") pod "network-check-target-whfmn" (UID: "8db714f3-333b-4b45-8e33-9a6b9c1c00e9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 21:22:38.523181 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.523094 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:38.523607 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:38.523253 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:38.904665 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:38.904589 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:39.113135 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:39.113096 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:39.306995 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:39.306898 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:39.523418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:39.523387 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:39.523418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:39.523415 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:39.523882 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:39.523496 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:39.523882 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:39.523643 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:39.710303 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:39.710210 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:40.327812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:40.327783 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:40.522660 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:40.522630 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:40.522845 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:40.522764 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:40.752838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:40.752762 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:40.942072 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:40.942038 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:41.522649 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:41.522615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:41.522840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:41.522665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:41.522840 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:41.522748 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:41.522962 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:41.522851 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:41.523727 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:41.523699 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:41.623580 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:41.623520 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:41.918564 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:41.918460 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:42.114713 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:42.114662 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:42.317886 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:42.317803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:42.522861 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:42.522840 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:42.522981 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:42.522958 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:42.692099 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:42.692066 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerStarted","Data":"5b0c1db78dbf0ea6501a2af452e9608f662ac830b9efcddbd0809954dd7d1045"} Jul 13 21:22:43.523094 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:43.523057 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:43.523535 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:43.523177 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:43.523535 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:43.523206 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:43.523535 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:43.523264 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:43.696817 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:43.696777 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="5b0c1db78dbf0ea6501a2af452e9608f662ac830b9efcddbd0809954dd7d1045" exitCode=0 Jul 13 21:22:43.696973 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:43.696826 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"5b0c1db78dbf0ea6501a2af452e9608f662ac830b9efcddbd0809954dd7d1045"} Jul 13 21:22:44.523423 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.523385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:44.523913 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:44.523471 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:44.701396 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.701366 2564 generic.go:356] "Generic (PLEG): container finished" podID="d66c5288-d9bf-4cb3-9321-7308fcf01ae3" containerID="c53bf2527fcb4af16b775b391548e74e50122bfa65e4261b6c0ea7cab7246cdf" exitCode=0 Jul 13 21:22:44.701543 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.701406 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerDied","Data":"c53bf2527fcb4af16b775b391548e74e50122bfa65e4261b6c0ea7cab7246cdf"} Jul 13 21:22:44.705691 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.705666 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:44.807993 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.807960 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:44.869893 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:44.869856 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:45.002784 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.002754 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:45.358957 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.358925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:45.458106 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.458074 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:45.523134 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.523094 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:45.523283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.523147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:45.523283 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:45.523222 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:45.523359 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:45.523337 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:45.706327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.706242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" event={"ID":"d66c5288-d9bf-4cb3-9321-7308fcf01ae3","Type":"ContainerStarted","Data":"ab47aaec813225f11e3a2909826fec931eeb9ed81899212f941483d54a5efbfe"} Jul 13 21:22:45.732320 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.732278 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tlk2k" podStartSLOduration=8.891554581 podStartE2EDuration="40.732261406s" totalImagesPullingTime="31.840706825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:45.731211385 +0000 UTC m=+41.785745972" watchObservedRunningTime="2026-07-13 21:22:45.732261406 +0000 UTC m=+41.786795992" Jul 13 21:22:45.773879 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.773823 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:45.835283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:45.835246 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:46.011807 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:46.011718 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:46.154896 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:46.154839 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:46.255052 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:46.255025 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:46.523416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:46.523391 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:46.523590 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:46.523483 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:46.585257 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:46.585227 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:47.160143 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:47.160088 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:47.415152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:47.415056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:47.523485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:47.523454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:47.523485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:47.523487 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:47.523682 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:47.523559 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:47.523682 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:47.523670 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:48.522774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:48.522741 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:48.523160 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:48.522854 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:49.522493 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:49.522454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:49.522735 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:49.522574 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:49.522735 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:49.522624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:49.522735 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:49.522705 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:50.522842 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:50.522810 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:50.523239 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:50.522941 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:51.522743 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:51.522707 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:51.522941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:51.522708 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:51.522941 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:51.522827 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:51.522941 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:51.522889 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:51.998072 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:51.997988 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:52.169973 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:52.169940 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:52.418524 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:52.418486 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:52.523351 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:52.523332 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:52.523688 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:52.523436 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:52.624912 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:52.624881 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:52.738276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:52.738154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:53.522542 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.522518 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:53.522673 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:53.522621 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:53.522673 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.522649 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:53.522747 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:53.522703 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:53.523437 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.523415 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:53.626601 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.626576 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:53.672531 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.672499 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffpgk"] Jul 13 21:22:53.672701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.672660 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:53.672820 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:53.672792 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:53.676636 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.676607 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-whfmn"] Jul 13 21:22:53.677248 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.677225 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x4nrq"] Jul 13 21:22:53.723524 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.723458 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:53.723653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.723458 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:53.723653 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:53.723576 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:53.723653 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:53.723614 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:53.731640 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:53.731607 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:54.193074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:54.193046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:54.264646 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:54.264617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:55.523572 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:55.523381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:55.523952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:55.523383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:55.523952 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:55.523656 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:55.523952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:55.523426 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:55.523952 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:55.523718 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:55.523952 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:55.523811 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:57.523376 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:57.523338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:57.523789 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:57.523338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:57.523789 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:57.523470 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x4nrq" podUID="ace0c1d9-d91d-40b4-b159-3996fbd78eff" Jul 13 21:22:57.523789 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:57.523350 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:57.523789 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:57.523533 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-whfmn" podUID="8db714f3-333b-4b45-8e33-9a6b9c1c00e9" Jul 13 21:22:57.523789 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:22:57.523601 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffpgk" podUID="5209694d-9769-4c3f-be50-c900e8628640" Jul 13 21:22:57.577898 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:57.577867 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:57.650281 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:57.650253 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:58.290253 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.290225 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-109.ec2.internal" event="NodeReady" Jul 13 21:22:58.290416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.290357 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 21:22:58.336584 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.336548 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-98wz5"] Jul 13 21:22:58.351674 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.351645 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2hv"] Jul 13 21:22:58.351823 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.351765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.354081 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.353984 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 21:22:58.354081 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.354028 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hgkt5\"" Jul 13 21:22:58.354081 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.354029 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 21:22:58.354377 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.354361 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 21:22:58.362451 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.362432 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-98wz5"] Jul 13 21:22:58.362565 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.362458 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2hv"] Jul 13 21:22:58.362565 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.362493 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b5m5z"] Jul 13 21:22:58.362634 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.362562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.366589 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.366567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fxsvw\"" Jul 13 21:22:58.366589 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.366587 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 21:22:58.366759 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.366573 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 21:22:58.366759 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.366587 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 21:22:58.366759 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.366567 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 21:22:58.378768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.378744 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5m5z"] Jul 13 21:22:58.378880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.378805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.380900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.380879 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 21:22:58.380900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.380892 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d9ws2\"" Jul 13 21:22:58.381068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.380946 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 21:22:58.433509 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08597cb4-981d-487d-a5a4-893aa674ea05-metrics-tls\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.433509 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee17311c-3732-451b-aa58-6dc841415566-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.433701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433547 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pfzj\" (UniqueName: \"kubernetes.io/projected/74c6ecbc-f0ec-4696-8965-8e142fc71f51-kube-api-access-2pfzj\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.433701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee17311c-3732-451b-aa58-6dc841415566-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.433701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08597cb4-981d-487d-a5a4-893aa674ea05-tmp-dir\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.433701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hdnw\" (UniqueName: \"kubernetes.io/projected/08597cb4-981d-487d-a5a4-893aa674ea05-kube-api-access-5hdnw\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.433701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74c6ecbc-f0ec-4696-8965-8e142fc71f51-cert\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.433852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee17311c-3732-451b-aa58-6dc841415566-crio-socket\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.433852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sld99\" (UniqueName: \"kubernetes.io/projected/ee17311c-3732-451b-aa58-6dc841415566-kube-api-access-sld99\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.433852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08597cb4-981d-487d-a5a4-893aa674ea05-config-volume\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.433852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.433829 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17311c-3732-451b-aa58-6dc841415566-data-volume\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.475047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.475016 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:58.534136 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08597cb4-981d-487d-a5a4-893aa674ea05-metrics-tls\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.534136 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee17311c-3732-451b-aa58-6dc841415566-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pfzj\" (UniqueName: \"kubernetes.io/projected/74c6ecbc-f0ec-4696-8965-8e142fc71f51-kube-api-access-2pfzj\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee17311c-3732-451b-aa58-6dc841415566-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08597cb4-981d-487d-a5a4-893aa674ea05-tmp-dir\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hdnw\" (UniqueName: \"kubernetes.io/projected/08597cb4-981d-487d-a5a4-893aa674ea05-kube-api-access-5hdnw\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74c6ecbc-f0ec-4696-8965-8e142fc71f51-cert\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee17311c-3732-451b-aa58-6dc841415566-crio-socket\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534440 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sld99\" (UniqueName: \"kubernetes.io/projected/ee17311c-3732-451b-aa58-6dc841415566-kube-api-access-sld99\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534461 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08597cb4-981d-487d-a5a4-893aa674ea05-config-volume\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17311c-3732-451b-aa58-6dc841415566-data-volume\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534597 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee17311c-3732-451b-aa58-6dc841415566-crio-socket\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08597cb4-981d-487d-a5a4-893aa674ea05-tmp-dir\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.534714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534712 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee17311c-3732-451b-aa58-6dc841415566-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.535180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.534846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee17311c-3732-451b-aa58-6dc841415566-data-volume\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.535180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.535064 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08597cb4-981d-487d-a5a4-893aa674ea05-config-volume\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.536986 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.536964 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:22:58.538550 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.538530 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08597cb4-981d-487d-a5a4-893aa674ea05-metrics-tls\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.538619 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.538534 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee17311c-3732-451b-aa58-6dc841415566-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.538619 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.538573 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74c6ecbc-f0ec-4696-8965-8e142fc71f51-cert\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.542163 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.542093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hdnw\" (UniqueName: \"kubernetes.io/projected/08597cb4-981d-487d-a5a4-893aa674ea05-kube-api-access-5hdnw\") pod \"dns-default-b5m5z\" (UID: \"08597cb4-981d-487d-a5a4-893aa674ea05\") " pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.542242 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.542163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sld99\" (UniqueName: \"kubernetes.io/projected/ee17311c-3732-451b-aa58-6dc841415566-kube-api-access-sld99\") pod \"insights-runtime-extractor-nx2hv\" (UID: \"ee17311c-3732-451b-aa58-6dc841415566\") " pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.542318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.542296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pfzj\" (UniqueName: \"kubernetes.io/projected/74c6ecbc-f0ec-4696-8965-8e142fc71f51-kube-api-access-2pfzj\") pod \"ingress-canary-98wz5\" (UID: \"74c6ecbc-f0ec-4696-8965-8e142fc71f51\") " pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.661799 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.661763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-98wz5" Jul 13 21:22:58.669967 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.669942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nx2hv" Jul 13 21:22:58.687172 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.687134 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5m5z" Jul 13 21:22:58.870562 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.870528 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-98wz5"] Jul 13 21:22:58.875028 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.875002 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2hv"] Jul 13 21:22:58.879802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:58.879756 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5m5z"] Jul 13 21:22:58.880535 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:58.880507 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee17311c_3732_451b_aa58_6dc841415566.slice/crio-840cbb47835dfaad848bb48f7a16a53a2554d5fb5a6dd5db78af2d2af51231e9 WatchSource:0}: Error finding container 840cbb47835dfaad848bb48f7a16a53a2554d5fb5a6dd5db78af2d2af51231e9: Status 404 returned error can't find the container with id 840cbb47835dfaad848bb48f7a16a53a2554d5fb5a6dd5db78af2d2af51231e9 Jul 13 21:22:58.883077 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:22:58.883056 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08597cb4_981d_487d_a5a4_893aa674ea05.slice/crio-69c4d941510c007db6811c3cad3448843dc7eb35263bd25f716db278ae452e58 WatchSource:0}: Error finding container 69c4d941510c007db6811c3cad3448843dc7eb35263bd25f716db278ae452e58: Status 404 returned error can't find the container with id 69c4d941510c007db6811c3cad3448843dc7eb35263bd25f716db278ae452e58 Jul 13 21:22:59.523590 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.523555 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:22:59.524035 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.524019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:22:59.524545 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.524490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:22:59.529077 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.528588 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 21:22:59.529077 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.528837 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 21:22:59.530020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.529450 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tk42z\"" Jul 13 21:22:59.530020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.529659 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r54dj\"" Jul 13 21:22:59.530020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.529867 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 21:22:59.532297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.532111 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 21:22:59.743997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.743957 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2hv" event={"ID":"ee17311c-3732-451b-aa58-6dc841415566","Type":"ContainerStarted","Data":"840cbb47835dfaad848bb48f7a16a53a2554d5fb5a6dd5db78af2d2af51231e9"} Jul 13 21:22:59.746818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.746755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-98wz5" event={"ID":"74c6ecbc-f0ec-4696-8965-8e142fc71f51","Type":"ContainerStarted","Data":"7071fd764de954a5542a06d455b96a1b09ad647ee260dd7e78e81f31b3b114d3"} Jul 13 21:22:59.748747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:22:59.748718 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5m5z" event={"ID":"08597cb4-981d-487d-a5a4-893aa674ea05","Type":"ContainerStarted","Data":"69c4d941510c007db6811c3cad3448843dc7eb35263bd25f716db278ae452e58"} Jul 13 21:23:00.285234 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:00.285177 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:00.413175 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:00.413096 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:01.193130 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:01.193080 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:01.292159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:01.292105 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:01.628299 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:01.628259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:01.710516 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:01.710450 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:02.399873 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.399838 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:23:02.404015 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.403988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.406474 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.406448 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 21:23:02.407567 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407400 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k8mjg\"" Jul 13 21:23:02.407567 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407513 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 21:23:02.407567 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407544 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 21:23:02.407783 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407606 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 21:23:02.407783 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407517 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 21:23:02.407892 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407852 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 21:23:02.407892 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.407853 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 21:23:02.412323 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.411921 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 21:23:02.413701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.413677 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:23:02.467095 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467095 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467342 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467245 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467342 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467342 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467311 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467459 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.467459 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.467395 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hbfs\" (UniqueName: \"kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.568760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.568760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568849 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hbfs\" (UniqueName: \"kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568890 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.568916 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569785 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.569717 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569785 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.569740 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569785 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.569719 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.569994 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.569834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.572536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.572491 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.572536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.572517 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.576944 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.576918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hbfs\" (UniqueName: \"kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs\") pod \"console-995d876b8-rtcmt\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:02.715518 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:02.715427 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:03.736100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.736061 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:03.778646 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.778613 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jp4kz"] Jul 13 21:23:03.782037 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.782015 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.785997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.785970 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 21:23:03.786140 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.785999 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 21:23:03.786477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.786450 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 21:23:03.786477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.786472 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 21:23:03.787204 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.787182 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 21:23:03.787302 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.787213 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hftj5\"" Jul 13 21:23:03.791366 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.791344 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jp4kz"] Jul 13 21:23:03.814401 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.814369 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:03.881245 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.881220 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.881384 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.881268 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.881384 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.881336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/771a6d2e-c05e-4686-b555-9ec29f446787-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.881472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.881450 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc77q\" (UniqueName: \"kubernetes.io/projected/771a6d2e-c05e-4686-b555-9ec29f446787-kube-api-access-rc77q\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.902546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.902507 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:03.983333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.982692 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.983333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.982787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.983333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.982831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/771a6d2e-c05e-4686-b555-9ec29f446787-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.983333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.982858 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc77q\" (UniqueName: \"kubernetes.io/projected/771a6d2e-c05e-4686-b555-9ec29f446787-kube-api-access-rc77q\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.983941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.983812 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/771a6d2e-c05e-4686-b555-9ec29f446787-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.985330 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.985306 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.985870 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.985852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/771a6d2e-c05e-4686-b555-9ec29f446787-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:03.992406 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:03.992353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc77q\" (UniqueName: \"kubernetes.io/projected/771a6d2e-c05e-4686-b555-9ec29f446787-kube-api-access-rc77q\") pod \"prometheus-operator-64785b6675-jp4kz\" (UID: \"771a6d2e-c05e-4686-b555-9ec29f446787\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:04.037318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.037238 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:23:04.040811 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:04.040663 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c6a94f_e891_4581_a2aa_c3ad0db77934.slice/crio-0a617dc989543fc4fca59261ba2d6099ebb00ab6c4f066ed3afd804974e69959 WatchSource:0}: Error finding container 0a617dc989543fc4fca59261ba2d6099ebb00ab6c4f066ed3afd804974e69959: Status 404 returned error can't find the container with id 0a617dc989543fc4fca59261ba2d6099ebb00ab6c4f066ed3afd804974e69959 Jul 13 21:23:04.093253 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.093224 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" Jul 13 21:23:04.225252 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.225221 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jp4kz"] Jul 13 21:23:04.228194 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:04.228168 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771a6d2e_c05e_4686_b555_9ec29f446787.slice/crio-dec4744a08af326ad5a28b268d06e206f8f988398ce423ddcd1041f1beb9f43d WatchSource:0}: Error finding container dec4744a08af326ad5a28b268d06e206f8f988398ce423ddcd1041f1beb9f43d: Status 404 returned error can't find the container with id dec4744a08af326ad5a28b268d06e206f8f988398ce423ddcd1041f1beb9f43d Jul 13 21:23:04.607729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.607691 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:04.679364 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.679141 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kksxm" Jul 13 21:23:04.711017 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.710980 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:04.764411 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.764373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-98wz5" event={"ID":"74c6ecbc-f0ec-4696-8965-8e142fc71f51","Type":"ContainerStarted","Data":"d21dca3a1e86831850e943d33a279d329eddc6c344794fa82f4eea5ac9ce3b05"} Jul 13 21:23:04.765641 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.765614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" event={"ID":"771a6d2e-c05e-4686-b555-9ec29f446787","Type":"ContainerStarted","Data":"dec4744a08af326ad5a28b268d06e206f8f988398ce423ddcd1041f1beb9f43d"} Jul 13 21:23:04.767597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.767529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5m5z" event={"ID":"08597cb4-981d-487d-a5a4-893aa674ea05","Type":"ContainerStarted","Data":"80830b5ed623a236292df95874087da093109c18d0f39c68b4e5f81607c13fa9"} Jul 13 21:23:04.767597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.767579 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5m5z" event={"ID":"08597cb4-981d-487d-a5a4-893aa674ea05","Type":"ContainerStarted","Data":"a172a7bc77ec52ddb128ee7b7b1841775ff560cf9568e5f8606ecf52628c6fff"} Jul 13 21:23:04.767756 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.767611 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b5m5z" Jul 13 21:23:04.768958 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.768934 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995d876b8-rtcmt" event={"ID":"35c6a94f-e891-4581-a2aa-c3ad0db77934","Type":"ContainerStarted","Data":"0a617dc989543fc4fca59261ba2d6099ebb00ab6c4f066ed3afd804974e69959"} Jul 13 21:23:04.770566 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.770545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2hv" event={"ID":"ee17311c-3732-451b-aa58-6dc841415566","Type":"ContainerStarted","Data":"9e90ce6de5e58de26955e8e36fe4f41cb6de8ec801bebca12a55fe3d58a66d13"} Jul 13 21:23:04.778880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.778745 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-98wz5" podStartSLOduration=1.769195713 podStartE2EDuration="6.778727914s" totalImagesPullingTime="5.009532201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:04.77839804 +0000 UTC m=+60.832932630" watchObservedRunningTime="2026-07-13 21:23:04.778727914 +0000 UTC m=+60.833262568" Jul 13 21:23:04.861517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.861425 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:04.918498 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:04.918467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:05.776833 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:05.776784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2hv" event={"ID":"ee17311c-3732-451b-aa58-6dc841415566","Type":"ContainerStarted","Data":"8e72f902d1f4367cdcb4e3cf2b6fc1cc3f00b1ff5706f8b677ea22f03b61a127"} Jul 13 21:23:05.805738 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:05.805670 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:05.906189 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:05.906153 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:06.196516 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:06.196403 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:06.254683 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:06.254650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:07.540456 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:07.540425 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:07.644050 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:07.644010 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:08.786712 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.786676 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" event={"ID":"771a6d2e-c05e-4686-b555-9ec29f446787","Type":"ContainerStarted","Data":"f4fd0c4969b6bb59c8ae8928e6b962d13391b31d4fb097e914af63d7ad421e92"} Jul 13 21:23:08.787249 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.786744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" event={"ID":"771a6d2e-c05e-4686-b555-9ec29f446787","Type":"ContainerStarted","Data":"c7f65df17642a27ed90d4b1524a058b9887054b688f2dd53d2481939673ba059"} Jul 13 21:23:08.788258 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.788218 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995d876b8-rtcmt" event={"ID":"35c6a94f-e891-4581-a2aa-c3ad0db77934","Type":"ContainerStarted","Data":"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464"} Jul 13 21:23:08.790111 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.790093 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2hv" event={"ID":"ee17311c-3732-451b-aa58-6dc841415566","Type":"ContainerStarted","Data":"13deb1a1d561ebb9b343f7e843a9ef10b72cb50b9b21be7e9c9d9ca8c6a8d552"} Jul 13 21:23:08.806419 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.806362 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-jp4kz" podStartSLOduration=1.465894043 podStartE2EDuration="5.806346189s" totalImagesPullingTime="4.340452146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:08.805315457 +0000 UTC m=+64.859850046" watchObservedRunningTime="2026-07-13 21:23:08.806346189 +0000 UTC m=+64.860880753" Jul 13 21:23:08.806596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.806499 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b5m5z" podStartSLOduration=5.813510526 podStartE2EDuration="10.80649028s" totalImagesPullingTime="4.992979754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:04.797413147 +0000 UTC m=+60.851947736" watchObservedRunningTime="2026-07-13 21:23:08.80649028 +0000 UTC m=+64.861024868" Jul 13 21:23:08.807695 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.807669 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:08.824542 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.824207 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-995d876b8-rtcmt" podStartSLOduration=2.288751857 podStartE2EDuration="6.824187455s" totalImagesPullingTime="4.535435598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:08.822864843 +0000 UTC m=+64.877399430" watchObservedRunningTime="2026-07-13 21:23:08.824187455 +0000 UTC m=+64.878722044" Jul 13 21:23:08.839522 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.839461 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nx2hv" podStartSLOduration=1.314008012 podStartE2EDuration="10.839437489s" totalImagesPullingTime="9.525429477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:08.838479305 +0000 UTC m=+64.893013900" watchObservedRunningTime="2026-07-13 21:23:08.839437489 +0000 UTC m=+64.893972076" Jul 13 21:23:08.901860 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:08.901822 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:09.002942 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:09.002897 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:09.103140 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:09.103033 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:10.233090 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.233035 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:23:10.233090 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.233100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:23:10.235609 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.235592 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 21:23:10.236225 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.236211 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 21:23:10.246259 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.246230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5209694d-9769-4c3f-be50-c900e8628640-original-pull-secret\") pod \"global-pull-secret-syncer-ffpgk\" (UID: \"5209694d-9769-4c3f-be50-c900e8628640\") " pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:23:10.246397 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.246261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ace0c1d9-d91d-40b4-b159-3996fbd78eff-metrics-certs\") pod \"network-metrics-daemon-x4nrq\" (UID: \"ace0c1d9-d91d-40b4-b159-3996fbd78eff\") " pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:23:10.333405 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.333373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:23:10.335812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.335791 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 21:23:10.346216 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.346194 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 21:23:10.348386 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.348361 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:10.356508 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.356488 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfj96\" (UniqueName: \"kubernetes.io/projected/8db714f3-333b-4b45-8e33-9a6b9c1c00e9-kube-api-access-pfj96\") pod \"network-check-target-whfmn\" (UID: \"8db714f3-333b-4b45-8e33-9a6b9c1c00e9\") " pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:23:10.361184 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.361160 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tk42z\"" Jul 13 21:23:10.366026 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.366013 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x4nrq" Jul 13 21:23:10.369259 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.369244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffpgk" Jul 13 21:23:10.383532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.383510 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r54dj\"" Jul 13 21:23:10.391540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.391509 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:23:10.426428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.426160 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:10.511472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.511439 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x4nrq"] Jul 13 21:23:10.514765 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:10.514731 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace0c1d9_d91d_40b4_b159_3996fbd78eff.slice/crio-444d04f2ab687b63b6032b69628049adca9ad3209e0466b52c2b0d3c8489f67c WatchSource:0}: Error finding container 444d04f2ab687b63b6032b69628049adca9ad3209e0466b52c2b0d3c8489f67c: Status 404 returned error can't find the container with id 444d04f2ab687b63b6032b69628049adca9ad3209e0466b52c2b0d3c8489f67c Jul 13 21:23:10.527385 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.527366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffpgk"] Jul 13 21:23:10.529785 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:10.529760 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5209694d_9769_4c3f_be50_c900e8628640.slice/crio-42b9f442077fceedad9faa997bc4bcbc67066ab124cb817c65876eec50f86ba8 WatchSource:0}: Error finding container 42b9f442077fceedad9faa997bc4bcbc67066ab124cb817c65876eec50f86ba8: Status 404 returned error can't find the container with id 42b9f442077fceedad9faa997bc4bcbc67066ab124cb817c65876eec50f86ba8 Jul 13 21:23:10.547339 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.547314 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-whfmn"] Jul 13 21:23:10.550842 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:10.550818 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db714f3_333b_4b45_8e33_9a6b9c1c00e9.slice/crio-a17ffe3ac4e8abac635035c57278ace5d9c2178f791361fb1d747631bdd20fb6 WatchSource:0}: Error finding container a17ffe3ac4e8abac635035c57278ace5d9c2178f791361fb1d747631bdd20fb6: Status 404 returned error can't find the container with id a17ffe3ac4e8abac635035c57278ace5d9c2178f791361fb1d747631bdd20fb6 Jul 13 21:23:10.798439 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.798403 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-whfmn" event={"ID":"8db714f3-333b-4b45-8e33-9a6b9c1c00e9","Type":"ContainerStarted","Data":"a17ffe3ac4e8abac635035c57278ace5d9c2178f791361fb1d747631bdd20fb6"} Jul 13 21:23:10.799561 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.799534 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffpgk" event={"ID":"5209694d-9769-4c3f-be50-c900e8628640","Type":"ContainerStarted","Data":"42b9f442077fceedad9faa997bc4bcbc67066ab124cb817c65876eec50f86ba8"} Jul 13 21:23:10.800505 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.800485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x4nrq" event={"ID":"ace0c1d9-d91d-40b4-b159-3996fbd78eff","Type":"ContainerStarted","Data":"444d04f2ab687b63b6032b69628049adca9ad3209e0466b52c2b0d3c8489f67c"} Jul 13 21:23:10.876772 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:10.876738 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.089207 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.089109 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.113494 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.112594 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl"] Jul 13 21:23:11.116360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.116316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.119471 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.118788 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 21:23:11.119471 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.119108 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p9rr9\"" Jul 13 21:23:11.119471 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.119330 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 21:23:11.127500 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.127423 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl"] Jul 13 21:23:11.131200 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.131175 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-89gnt"] Jul 13 21:23:11.135191 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.134703 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.139020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.138452 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 21:23:11.139020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.138658 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8rcwv\"" Jul 13 21:23:11.139020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.138782 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 21:23:11.139020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.138877 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 21:23:11.175727 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.175662 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.239367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.239367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239262 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-metrics-client-ca\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.239367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-root\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.239367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239334 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.239367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239361 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxtq8\" (UniqueName: \"kubernetes.io/projected/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-kube-api-access-qxtq8\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-textfile\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-accelerators-collector-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239552 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-sys\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-wtmp\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239638 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.240012 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.239677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr8fr\" (UniqueName: \"kubernetes.io/projected/2f98f5c5-bf03-4e64-865b-fd1db772aee8-kube-api-access-cr8fr\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.241686 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.241640 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.325508 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.325400 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-root\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340296 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxtq8\" (UniqueName: \"kubernetes.io/projected/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-kube-api-access-qxtq8\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-textfile\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340349 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-root\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:11.340458 2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:11.340519 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls podName:f7e4ee8d-c771-445e-93fb-d46ab7b7d234 nodeName:}" failed. No retries permitted until 2026-07-13 21:23:11.840497948 +0000 UTC m=+67.895032515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-85zgl" (UID: "f7e4ee8d-c771-445e-93fb-d46ab7b7d234") : secret "openshift-state-metrics-tls" not found Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-accelerators-collector-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.340362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-accelerators-collector-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341080 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-sys\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-wtmp\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-textfile\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.341955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.341955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-sys\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.341959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-wtmp\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342035 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.342178 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342066 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342178 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr8fr\" (UniqueName: \"kubernetes.io/projected/2f98f5c5-bf03-4e64-865b-fd1db772aee8-kube-api-access-cr8fr\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342252 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342252 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-metrics-client-ca\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.342716 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.342693 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f98f5c5-bf03-4e64-865b-fd1db772aee8-metrics-client-ca\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.343413 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:11.343004 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 21:23:11.343413 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:11.343058 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls podName:2f98f5c5-bf03-4e64-865b-fd1db772aee8 nodeName:}" failed. No retries permitted until 2026-07-13 21:23:11.843042062 +0000 UTC m=+67.897576630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls") pod "node-exporter-89gnt" (UID: "2f98f5c5-bf03-4e64-865b-fd1db772aee8") : secret "node-exporter-tls" not found Jul 13 21:23:11.345265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.345224 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.346386 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.346355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.353168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.352688 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxtq8\" (UniqueName: \"kubernetes.io/projected/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-kube-api-access-qxtq8\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.353168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.353128 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr8fr\" (UniqueName: \"kubernetes.io/projected/2f98f5c5-bf03-4e64-865b-fd1db772aee8-kube-api-access-cr8fr\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.473907 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.473792 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.575078 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.575047 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.675355 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.675312 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.788674 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.788620 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:11.807388 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.807348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x4nrq" event={"ID":"ace0c1d9-d91d-40b4-b159-3996fbd78eff","Type":"ContainerStarted","Data":"42137f3d1273c076a308b77d5df11c414f0460e62eee8c8c6ae3fd900ca91012"} Jul 13 21:23:11.807534 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.807443 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x4nrq" event={"ID":"ace0c1d9-d91d-40b4-b159-3996fbd78eff","Type":"ContainerStarted","Data":"9975747ede43b0aa63a0b3aca43b9ffc46017d91228ab56a553ff5303fea9f47"} Jul 13 21:23:11.837854 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.836722 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x4nrq" podStartSLOduration=65.778298417 podStartE2EDuration="1m6.836702164s" totalImagesPullingTime="1.058403747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:11.835912192 +0000 UTC m=+67.890446782" watchObservedRunningTime="2026-07-13 21:23:11.836702164 +0000 UTC m=+67.891236750" Jul 13 21:23:11.846991 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.846959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.847182 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.847018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:11.849994 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.849933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f98f5c5-bf03-4e64-865b-fd1db772aee8-node-exporter-tls\") pod \"node-exporter-89gnt\" (UID: \"2f98f5c5-bf03-4e64-865b-fd1db772aee8\") " pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:11.850133 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:11.850097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f7e4ee8d-c771-445e-93fb-d46ab7b7d234-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-85zgl\" (UID: \"f7e4ee8d-c771-445e-93fb-d46ab7b7d234\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:12.030476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.030434 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" Jul 13 21:23:12.048240 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.048000 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89gnt" Jul 13 21:23:12.065737 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:12.065689 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f98f5c5_bf03_4e64_865b_fd1db772aee8.slice/crio-44ea6d851faba619482cb8157ddda5c089cc7118546c73989413ad2e61c3bec1 WatchSource:0}: Error finding container 44ea6d851faba619482cb8157ddda5c089cc7118546c73989413ad2e61c3bec1: Status 404 returned error can't find the container with id 44ea6d851faba619482cb8157ddda5c089cc7118546c73989413ad2e61c3bec1 Jul 13 21:23:12.091187 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.091135 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.186962 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.186899 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.203422 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.203395 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl"] Jul 13 21:23:12.207506 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:12.207463 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e4ee8d_c771_445e_93fb_d46ab7b7d234.slice/crio-35d54d79cf34b2e4db4d0158c15aaa42f5fce8b06c52db2471bb368ce8b81d71 WatchSource:0}: Error finding container 35d54d79cf34b2e4db4d0158c15aaa42f5fce8b06c52db2471bb368ce8b81d71: Status 404 returned error can't find the container with id 35d54d79cf34b2e4db4d0158c15aaa42f5fce8b06c52db2471bb368ce8b81d71 Jul 13 21:23:12.282279 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.282039 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.382431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.382394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.601437 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.601384 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.661760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.661683 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.715829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.715795 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:12.715986 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.715859 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:12.721503 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.721481 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:12.725737 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.725703 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:12.812389 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.812340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89gnt" event={"ID":"2f98f5c5-bf03-4e64-865b-fd1db772aee8","Type":"ContainerStarted","Data":"44ea6d851faba619482cb8157ddda5c089cc7118546c73989413ad2e61c3bec1"} Jul 13 21:23:12.814723 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.814683 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" event={"ID":"f7e4ee8d-c771-445e-93fb-d46ab7b7d234","Type":"ContainerStarted","Data":"8008d5c11d793d07da696bc7bf9099d6d6f5b7c85a5712895b20e3038bbf6721"} Jul 13 21:23:12.814855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.814750 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" event={"ID":"f7e4ee8d-c771-445e-93fb-d46ab7b7d234","Type":"ContainerStarted","Data":"33e66294cde04b2b118eb8c84e58662bd8c6c3d39dca23607140b938ffd55816"} Jul 13 21:23:12.814855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.814783 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" event={"ID":"f7e4ee8d-c771-445e-93fb-d46ab7b7d234","Type":"ContainerStarted","Data":"35d54d79cf34b2e4db4d0158c15aaa42f5fce8b06c52db2471bb368ce8b81d71"} Jul 13 21:23:12.819660 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.819625 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:12.980985 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:12.980890 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:13.079656 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:13.079616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:13.180652 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:13.180616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:14.119394 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:14.119296 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:14.220546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:14.220510 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:14.401345 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:14.401266 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:14.503460 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:14.503389 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:14.780215 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:14.780179 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b5m5z" Jul 13 21:23:15.522957 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.522918 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d8d59bc8b-znns2"] Jul 13 21:23:15.526134 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.526097 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.529655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529433 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 21:23:15.529655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529435 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 21:23:15.529655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529495 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 21:23:15.529655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529506 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 21:23:15.529655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529598 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6s5hw\"" Jul 13 21:23:15.529988 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.529793 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4t6ss3ior9n1m\"" Jul 13 21:23:15.535798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.535779 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8d59bc8b-znns2"] Jul 13 21:23:15.680174 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mpql\" (UniqueName: \"kubernetes.io/projected/69fbd780-2856-42d8-acb3-2fa4b8edffb9-kube-api-access-7mpql\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-tls\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-client-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680318 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69fbd780-2856-42d8-acb3-2fa4b8edffb9-audit-log\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680352 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-client-certs\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.680574 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.680420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-metrics-server-audit-profiles\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781202 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-client-certs\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-metrics-server-audit-profiles\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781268 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mpql\" (UniqueName: \"kubernetes.io/projected/69fbd780-2856-42d8-acb3-2fa4b8edffb9-kube-api-access-7mpql\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781443 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-tls\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781443 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-client-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.781443 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69fbd780-2856-42d8-acb3-2fa4b8edffb9-audit-log\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.782329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.781805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/69fbd780-2856-42d8-acb3-2fa4b8edffb9-audit-log\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.782329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.782280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.783032 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.782991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/69fbd780-2856-42d8-acb3-2fa4b8edffb9-metrics-server-audit-profiles\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.784723 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.784701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-client-ca-bundle\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.786675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.786650 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-tls\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.786772 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.786735 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/69fbd780-2856-42d8-acb3-2fa4b8edffb9-secret-metrics-server-client-certs\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.791486 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.791458 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mpql\" (UniqueName: \"kubernetes.io/projected/69fbd780-2856-42d8-acb3-2fa4b8edffb9-kube-api-access-7mpql\") pod \"metrics-server-5d8d59bc8b-znns2\" (UID: \"69fbd780-2856-42d8-acb3-2fa4b8edffb9\") " pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.829348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.829320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-whfmn" event={"ID":"8db714f3-333b-4b45-8e33-9a6b9c1c00e9","Type":"ContainerStarted","Data":"785052dbfb666c5f7f89c03a68d1b5643370b22a1024e32ce1dcfa1d80c4ccde"} Jul 13 21:23:15.829464 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.829443 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:23:15.837134 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.837095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:15.846780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.846462 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-whfmn" podStartSLOduration=65.690414399 podStartE2EDuration="1m10.84644574s" totalImagesPullingTime="5.156031341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:15.845382858 +0000 UTC m=+71.899917461" watchObservedRunningTime="2026-07-13 21:23:15.84644574 +0000 UTC m=+71.900980327" Jul 13 21:23:15.921594 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.921557 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:15.939716 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.939687 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6"] Jul 13 21:23:15.948484 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.948460 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:15.952101 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.951389 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-f5dvl\"" Jul 13 21:23:15.952101 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.951890 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 21:23:15.953845 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.953781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6"] Jul 13 21:23:15.996277 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:15.996251 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d8d59bc8b-znns2"] Jul 13 21:23:15.998935 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:15.998908 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69fbd780_2856_42d8_acb3_2fa4b8edffb9.slice/crio-7b6c6a482658a27aa3a2e81d9df49dda896ca5ced21abe1abcb983add9b0fc8d WatchSource:0}: Error finding container 7b6c6a482658a27aa3a2e81d9df49dda896ca5ced21abe1abcb983add9b0fc8d: Status 404 returned error can't find the container with id 7b6c6a482658a27aa3a2e81d9df49dda896ca5ced21abe1abcb983add9b0fc8d Jul 13 21:23:16.001351 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.001320 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:16.086373 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.086331 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqhrn\" (UniqueName: \"kubernetes.io/projected/0ece67f8-49ce-48d1-996d-8484567109e1-kube-api-access-lqhrn\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.086624 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.086381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.132360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.132323 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:16.187699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.187663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqhrn\" (UniqueName: \"kubernetes.io/projected/0ece67f8-49ce-48d1-996d-8484567109e1-kube-api-access-lqhrn\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.187870 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.187722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.187870 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:16.187851 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 13 21:23:16.187953 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:16.187919 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert podName:0ece67f8-49ce-48d1-996d-8484567109e1 nodeName:}" failed. No retries permitted until 2026-07-13 21:23:16.687897227 +0000 UTC m=+72.742431810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-lvtd6" (UID: "0ece67f8-49ce-48d1-996d-8484567109e1") : secret "monitoring-plugin-cert" not found Jul 13 21:23:16.196063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.196035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqhrn\" (UniqueName: \"kubernetes.io/projected/0ece67f8-49ce-48d1-996d-8484567109e1-kube-api-access-lqhrn\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.225895 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.225860 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:16.330852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.330822 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cd9488bbb-nqzst"] Jul 13 21:23:16.335529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.335509 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.338403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338369 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gshfb\"" Jul 13 21:23:16.338403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338384 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 21:23:16.338403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338393 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 21:23:16.338702 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338376 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 21:23:16.338702 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338369 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 21:23:16.338702 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.338376 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 21:23:16.343919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.343901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 21:23:16.348320 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.348297 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cd9488bbb-nqzst"] Jul 13 21:23:16.490784 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-metrics-client-ca\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.490784 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-federate-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-serving-certs-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqdmc\" (UniqueName: \"kubernetes.io/projected/be01b040-2b51-4664-a2ac-823f1a060d9d-kube-api-access-fqdmc\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.490990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.491203 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.491023 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.551030 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.550988 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:16.592113 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592113 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592113 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592111 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592192 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-metrics-client-ca\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-federate-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-serving-certs-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.592506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.592341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqdmc\" (UniqueName: \"kubernetes.io/projected/be01b040-2b51-4664-a2ac-823f1a060d9d-kube-api-access-fqdmc\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.593427 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.593375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-serving-certs-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.593427 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.593404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-metrics-client-ca\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.593617 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.593404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.595798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.595733 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-telemeter-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.595798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.595736 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.595964 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.595859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-secret-telemeter-client\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.595964 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.595864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be01b040-2b51-4664-a2ac-823f1a060d9d-federate-client-tls\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.601157 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.601107 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqdmc\" (UniqueName: \"kubernetes.io/projected/be01b040-2b51-4664-a2ac-823f1a060d9d-kube-api-access-fqdmc\") pod \"telemeter-client-6cd9488bbb-nqzst\" (UID: \"be01b040-2b51-4664-a2ac-823f1a060d9d\") " pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.645037 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.645007 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" Jul 13 21:23:16.693098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.693061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.696272 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.696241 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0ece67f8-49ce-48d1-996d-8484567109e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lvtd6\" (UID: \"0ece67f8-49ce-48d1-996d-8484567109e1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:16.792532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.792498 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cd9488bbb-nqzst"] Jul 13 21:23:16.795259 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:16.795229 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe01b040_2b51_4664_a2ac_823f1a060d9d.slice/crio-ae101aafb0b43d9866e6d36923379624fae70e320e874ebbcd02a0c12d0ed5a9 WatchSource:0}: Error finding container ae101aafb0b43d9866e6d36923379624fae70e320e874ebbcd02a0c12d0ed5a9: Status 404 returned error can't find the container with id ae101aafb0b43d9866e6d36923379624fae70e320e874ebbcd02a0c12d0ed5a9 Jul 13 21:23:16.837886 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.837844 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffpgk" event={"ID":"5209694d-9769-4c3f-be50-c900e8628640","Type":"ContainerStarted","Data":"d1ef71698a64c1ca648ebe13bd2c9d350243e44aeda2ec42c1612975bccdd2bb"} Jul 13 21:23:16.839686 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.839656 2564 generic.go:356] "Generic (PLEG): container finished" podID="2f98f5c5-bf03-4e64-865b-fd1db772aee8" containerID="83e2330f0b96264691c86049372be9422996fcc8513a901745e88c3e4dfce8ae" exitCode=0 Jul 13 21:23:16.839820 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.839746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89gnt" event={"ID":"2f98f5c5-bf03-4e64-865b-fd1db772aee8","Type":"ContainerDied","Data":"83e2330f0b96264691c86049372be9422996fcc8513a901745e88c3e4dfce8ae"} Jul 13 21:23:16.842224 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.842153 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" event={"ID":"f7e4ee8d-c771-445e-93fb-d46ab7b7d234","Type":"ContainerStarted","Data":"8099755bb00ea2d18824d601b342e5e6c536ddde9496008934d292c7aa57dc7f"} Jul 13 21:23:16.843581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.843559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" event={"ID":"be01b040-2b51-4664-a2ac-823f1a060d9d","Type":"ContainerStarted","Data":"ae101aafb0b43d9866e6d36923379624fae70e320e874ebbcd02a0c12d0ed5a9"} Jul 13 21:23:16.844964 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.844944 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" event={"ID":"69fbd780-2856-42d8-acb3-2fa4b8edffb9","Type":"ContainerStarted","Data":"7b6c6a482658a27aa3a2e81d9df49dda896ca5ced21abe1abcb983add9b0fc8d"} Jul 13 21:23:16.852620 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.852553 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ffpgk" podStartSLOduration=66.675332781 podStartE2EDuration="1m11.85253568s" totalImagesPullingTime="5.177202899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:16.851912478 +0000 UTC m=+72.906447065" watchObservedRunningTime="2026-07-13 21:23:16.85253568 +0000 UTC m=+72.907070269" Jul 13 21:23:16.884527 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.884484 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-85zgl" podStartSLOduration=2.603179486 podStartE2EDuration="5.884468822s" totalImagesPullingTime="3.281289336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:16.882914719 +0000 UTC m=+72.937449332" watchObservedRunningTime="2026-07-13 21:23:16.884468822 +0000 UTC m=+72.939003409" Jul 13 21:23:16.901816 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:16.901795 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:17.049733 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.049701 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6"] Jul 13 21:23:17.053611 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:23:17.053578 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ece67f8_49ce_48d1_996d_8484567109e1.slice/crio-00d94ff46ec25434df800dbcb15321cbe2599c7d68b72568d351605f57d3cbc8 WatchSource:0}: Error finding container 00d94ff46ec25434df800dbcb15321cbe2599c7d68b72568d351605f57d3cbc8: Status 404 returned error can't find the container with id 00d94ff46ec25434df800dbcb15321cbe2599c7d68b72568d351605f57d3cbc8 Jul 13 21:23:17.386343 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.386309 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:17.655282 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.655256 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:17.755801 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.755769 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:17.859014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.858974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89gnt" event={"ID":"2f98f5c5-bf03-4e64-865b-fd1db772aee8","Type":"ContainerStarted","Data":"92a1623279b555333d7251b8086349e997960b1ce550825608cb4700ac72f641"} Jul 13 21:23:17.859327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.859085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89gnt" event={"ID":"2f98f5c5-bf03-4e64-865b-fd1db772aee8","Type":"ContainerStarted","Data":"f17700103b42c698d6027491d4c34f08c6d02eca551ccecd92888131cc1d6061"} Jul 13 21:23:17.861343 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.861314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" event={"ID":"69fbd780-2856-42d8-acb3-2fa4b8edffb9","Type":"ContainerStarted","Data":"8d9d656538d80bfd460b1234d61f983d5f876dc63912b66f86043eaa81717b0c"} Jul 13 21:23:17.862769 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.862739 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" event={"ID":"0ece67f8-49ce-48d1-996d-8484567109e1","Type":"ContainerStarted","Data":"00d94ff46ec25434df800dbcb15321cbe2599c7d68b72568d351605f57d3cbc8"} Jul 13 21:23:17.883919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.883868 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-89gnt" podStartSLOduration=3.150600663 podStartE2EDuration="6.883849024s" totalImagesPullingTime="3.640808737s" totalInitContainerRuntime="92.439624ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:17.881368034 +0000 UTC m=+73.935902637" watchObservedRunningTime="2026-07-13 21:23:17.883849024 +0000 UTC m=+73.938383612" Jul 13 21:23:17.900449 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:17.900366 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" podStartSLOduration=1.327550103 podStartE2EDuration="2.900351417s" totalImagesPullingTime="1.572801314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:17.898999007 +0000 UTC m=+73.953533584" watchObservedRunningTime="2026-07-13 21:23:17.900351417 +0000 UTC m=+73.954886003" Jul 13 21:23:19.558909 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.558870 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:19.661254 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.661221 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:19.732921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.732892 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:19.816728 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.816703 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:19.873697 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.873660 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" event={"ID":"be01b040-2b51-4664-a2ac-823f1a060d9d","Type":"ContainerStarted","Data":"aa4c45440b96fdf071106c781134d52a19bf0e71ae673513dde592fae2b4ef6e"} Jul 13 21:23:19.875064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.875035 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" event={"ID":"0ece67f8-49ce-48d1-996d-8484567109e1","Type":"ContainerStarted","Data":"94d192f4ab031ad665719e7d74fa48782950b6ec4b3690929b36b08b1019d77e"} Jul 13 21:23:19.875270 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.875242 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:19.879995 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.879973 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" Jul 13 21:23:19.890939 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.890901 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lvtd6" podStartSLOduration=3.129357656 podStartE2EDuration="4.890887461s" totalImagesPullingTime="1.761529805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:19.890300665 +0000 UTC m=+75.944835251" watchObservedRunningTime="2026-07-13 21:23:19.890887461 +0000 UTC m=+75.945422048" Jul 13 21:23:19.895645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:19.895622 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:21.122344 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:21.122300 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:21.187211 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:21.187104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:21.884534 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:21.884496 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" event={"ID":"be01b040-2b51-4664-a2ac-823f1a060d9d","Type":"ContainerStarted","Data":"60c835b69ea88d649cb55899686f6de1a8792d4e973ef73b174a7e1b352d0424"} Jul 13 21:23:21.884717 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:21.884603 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" event={"ID":"be01b040-2b51-4664-a2ac-823f1a060d9d","Type":"ContainerStarted","Data":"f94a629cd085a9c3a9ceee29ba84c47f9028d5d21c5fe51122a8f20562667835"} Jul 13 21:23:21.907529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:21.907404 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cd9488bbb-nqzst" podStartSLOduration=1.8847999519999998 podStartE2EDuration="5.907385165s" totalImagesPullingTime="4.022585213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:21.906041041 +0000 UTC m=+77.960575631" watchObservedRunningTime="2026-07-13 21:23:21.907385165 +0000 UTC m=+77.961919755" Jul 13 21:23:22.855099 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:22.855064 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:22.952837 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:22.952796 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:23.453637 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:23.453599 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:23.555039 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:23.555010 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:23.926579 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:23.926544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:24.024856 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:24.024824 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:28.637042 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:28.636918 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:28.694955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:28.694922 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:31.144339 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:31.144308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:31.246931 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:31.246900 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:32.917790 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:32.917751 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:33.019284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:33.019252 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:33.251403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:33.251321 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:23:33.368407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:33.368367 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:33.468256 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:33.468226 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:33.569245 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:33.569205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:35.837955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:35.837918 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:35.838369 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:35.837987 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:41.876036 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:41.876006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-98wz5_74c6ecbc-f0ec-4696-8965-8e142fc71f51/serve-healthcheck-canary/0.log" Jul 13 21:23:43.371521 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:43.371489 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:46.165548 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:46.165511 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:46.259639 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:46.259606 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:46.847919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:46.847887 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-whfmn" Jul 13 21:23:53.255243 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:53.255207 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:53.356828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:53.356797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:53.456694 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:53.456652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:23:55.842702 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:55.842671 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:55.846475 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:55.846452 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d8d59bc8b-znns2" Jul 13 21:23:58.271831 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.271777 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-995d876b8-rtcmt" podUID="35c6a94f-e891-4581-a2aa-c3ad0db77934" containerName="console" containerID="cri-o://e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464" gracePeriod=15 Jul 13 21:23:58.518928 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.518904 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:58.612474 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612369 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612471 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612506 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612528 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hbfs\" (UniqueName: \"kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612553 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612584 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612626 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config\") pod \"35c6a94f-e891-4581-a2aa-c3ad0db77934\" (UID: \"35c6a94f-e891-4581-a2aa-c3ad0db77934\") " Jul 13 21:23:58.612957 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612778 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca" (OuterVolumeSpecName: "service-ca") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:23:58.613016 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.612965 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:23:58.613076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.613018 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:23:58.613275 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.613242 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config" (OuterVolumeSpecName: "console-config") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:23:58.614943 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.614921 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:23:58.615018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.614966 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs" (OuterVolumeSpecName: "kube-api-access-8hbfs") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "kube-api-access-8hbfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:23:58.615018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.614985 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "35c6a94f-e891-4581-a2aa-c3ad0db77934" (UID: "35c6a94f-e891-4581-a2aa-c3ad0db77934"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:23:58.713762 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713729 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.713762 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713755 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-serving-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.713762 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713765 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-trusted-ca-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.714007 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713774 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hbfs\" (UniqueName: \"kubernetes.io/projected/35c6a94f-e891-4581-a2aa-c3ad0db77934-kube-api-access-8hbfs\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.714007 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713783 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-oauth-config\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.714007 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713792 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-oauth-serving-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:58.714007 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:58.713800 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c6a94f-e891-4581-a2aa-c3ad0db77934-console-config\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:23:59.018026 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.017989 2564 generic.go:356] "Generic (PLEG): container finished" podID="35c6a94f-e891-4581-a2aa-c3ad0db77934" containerID="e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464" exitCode=0 Jul 13 21:23:59.018210 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.018036 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995d876b8-rtcmt" Jul 13 21:23:59.018210 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.018071 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995d876b8-rtcmt" event={"ID":"35c6a94f-e891-4581-a2aa-c3ad0db77934","Type":"ContainerDied","Data":"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464"} Jul 13 21:23:59.018210 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.018139 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995d876b8-rtcmt" event={"ID":"35c6a94f-e891-4581-a2aa-c3ad0db77934","Type":"ContainerDied","Data":"0a617dc989543fc4fca59261ba2d6099ebb00ab6c4f066ed3afd804974e69959"} Jul 13 21:23:59.018210 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.018166 2564 scope.go:117] "RemoveContainer" containerID="e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464" Jul 13 21:23:59.027168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.027151 2564 scope.go:117] "RemoveContainer" containerID="e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464" Jul 13 21:23:59.027423 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:23:59.027403 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464\": container with ID starting with e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464 not found: ID does not exist" containerID="e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464" Jul 13 21:23:59.027470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.027436 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464"} err="failed to get container status \"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464\": rpc error: code = NotFound desc = could not find container \"e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464\": container with ID starting with e772819de2fbb336811d0ead0183fdd326e0da0ccdf142665803d5f61b2a4464 not found: ID does not exist" Jul 13 21:23:59.037421 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.037391 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:23:59.040989 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:23:59.040967 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-995d876b8-rtcmt"] Jul 13 21:24:00.531187 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:00.531151 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c6a94f-e891-4581-a2aa-c3ad0db77934" path="/var/lib/kubelet/pods/35c6a94f-e891-4581-a2aa-c3ad0db77934/volumes" Jul 13 21:24:27.279103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:27.279062 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:27.343996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:27.343968 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:33.008380 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.008343 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 21:24:33.008864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.008602 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c6a94f-e891-4581-a2aa-c3ad0db77934" containerName="console" Jul 13 21:24:33.008864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.008614 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c6a94f-e891-4581-a2aa-c3ad0db77934" containerName="console" Jul 13 21:24:33.008864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.008661 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c6a94f-e891-4581-a2aa-c3ad0db77934" containerName="console" Jul 13 21:24:33.012281 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.012259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.015409 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015390 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kh2wb\"" Jul 13 21:24:33.015529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015391 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 21:24:33.015768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015673 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 21:24:33.015768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015673 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 21:24:33.015768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015726 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 21:24:33.015768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015734 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 21:24:33.015768 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015734 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 21:24:33.016006 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015774 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 21:24:33.016006 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.015790 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 21:24:33.020018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.020000 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 21:24:33.026167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.026148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 21:24:33.161202 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161202 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-web-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161250 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161341 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkb7\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-kube-api-access-2hkb7\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161369 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161430 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-out\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.161585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.161525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.208688 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.208655 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:24:33.211736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.211720 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.214068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.214049 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 21:24:33.214427 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.214413 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 21:24:33.214858 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.214845 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 21:24:33.215087 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.215073 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 21:24:33.215150 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.215104 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 21:24:33.215296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.215284 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 21:24:33.215807 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.215795 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 21:24:33.216086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.216069 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k8mjg\"" Jul 13 21:24:33.224882 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.224866 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 21:24:33.227744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.227726 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:24:33.262261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-web-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262261 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkb7\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-kube-api-access-2hkb7\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262338 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262395 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-out\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.262591 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.262523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.263332 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.263307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.264078 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.263686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.266182 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.266158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.266612 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.266585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.267497 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.267475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-out\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.270627 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.268996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.270627 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.270435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee0e473f-5618-45de-9a8a-8662cf1b831e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.270905 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.270881 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.271026 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.270993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.271344 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.271323 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.272188 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.272097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee0e473f-5618-45de-9a8a-8662cf1b831e-web-config\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.272335 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.272313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.272416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.272400 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkb7\" (UniqueName: \"kubernetes.io/projected/ee0e473f-5618-45de-9a8a-8662cf1b831e-kube-api-access-2hkb7\") pod \"alertmanager-main-0\" (UID: \"ee0e473f-5618-45de-9a8a-8662cf1b831e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.322510 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.322468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 21:24:33.363464 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363430 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363464 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363467 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363489 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58tms\" (UniqueName: \"kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.363736 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.363720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.376475 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.376413 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:33.431442 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.431406 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:33.465564 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465533 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.465815 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.465815 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58tms\" (UniqueName: \"kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.465815 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.466000 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465934 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.466000 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.465987 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.466212 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.466012 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.466400 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.466340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.467103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.466665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.467103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.467052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.467103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.467075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.467263 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.467188 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 21:24:33.468540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.468510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.468720 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.468696 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.468936 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:24:33.468915 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0e473f_5618_45de_9a8a_8662cf1b831e.slice/crio-ade748093763384b9a48a39a93d207732bcfdd725eadf2da9533d047a5ce1c57 WatchSource:0}: Error finding container ade748093763384b9a48a39a93d207732bcfdd725eadf2da9533d047a5ce1c57: Status 404 returned error can't find the container with id ade748093763384b9a48a39a93d207732bcfdd725eadf2da9533d047a5ce1c57 Jul 13 21:24:33.473592 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.473567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58tms\" (UniqueName: \"kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms\") pod \"console-696c6d8b9b-6wchx\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.521841 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.521818 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:33.647952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:33.647908 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:24:33.650375 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:24:33.650347 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530eecca_2abb_4271_b41c_d00a266daa91.slice/crio-49c30bcc9a69b25da0314260ecd975ad351f09e872bad23898337f3b99afa08d WatchSource:0}: Error finding container 49c30bcc9a69b25da0314260ecd975ad351f09e872bad23898337f3b99afa08d: Status 404 returned error can't find the container with id 49c30bcc9a69b25da0314260ecd975ad351f09e872bad23898337f3b99afa08d Jul 13 21:24:34.139100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.139062 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c6d8b9b-6wchx" event={"ID":"530eecca-2abb-4271-b41c-d00a266daa91","Type":"ContainerStarted","Data":"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247"} Jul 13 21:24:34.139543 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.139138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c6d8b9b-6wchx" event={"ID":"530eecca-2abb-4271-b41c-d00a266daa91","Type":"ContainerStarted","Data":"49c30bcc9a69b25da0314260ecd975ad351f09e872bad23898337f3b99afa08d"} Jul 13 21:24:34.140593 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.140568 2564 generic.go:356] "Generic (PLEG): container finished" podID="ee0e473f-5618-45de-9a8a-8662cf1b831e" containerID="aae085f8e7009a8478035488b26107a7e5589dffba846d5ad83d0ee598f8c1a3" exitCode=0 Jul 13 21:24:34.140675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.140614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerDied","Data":"aae085f8e7009a8478035488b26107a7e5589dffba846d5ad83d0ee598f8c1a3"} Jul 13 21:24:34.140675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.140659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"ade748093763384b9a48a39a93d207732bcfdd725eadf2da9533d047a5ce1c57"} Jul 13 21:24:34.158617 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:34.158570 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-696c6d8b9b-6wchx" podStartSLOduration=1.158554987 podStartE2EDuration="1.158554987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:24:34.157140446 +0000 UTC m=+150.211675025" watchObservedRunningTime="2026-07-13 21:24:34.158554987 +0000 UTC m=+150.213089574" Jul 13 21:24:36.153095 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:36.153059 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"cb2b492467d5d5b8b6d8466b38a656e1d91294c23677821345e446787e7f6de2"} Jul 13 21:24:36.153461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:36.153142 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"708bf9727111cfad6a8ee253a6856b4a8fa61281b855929850b5ed2b1456a30d"} Jul 13 21:24:36.153461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:36.153174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"1696afcb321dc09c58a1755fa11e9a8c08268c689d70fedab4e2810fc272263d"} Jul 13 21:24:36.153461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:36.153200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"90dbd07f87fb195c4092e3987c9aadcb5ff91b2adbc59010f4bfdc6587dc8cc9"} Jul 13 21:24:36.153461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:36.153224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"a0d15807c67d8eaf162adde888381f889a18d3508b05c6888bfa24495ebded84"} Jul 13 21:24:37.159947 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:37.159912 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee0e473f-5618-45de-9a8a-8662cf1b831e","Type":"ContainerStarted","Data":"cafe4268a0da04cadbf8898ef3f1cce216ff3866f773b0dfd2a40bea193a9323"} Jul 13 21:24:37.187450 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:37.187394 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.622353184 podStartE2EDuration="5.187372464s" totalImagesPullingTime="2.549663503s" totalInitContainerRuntime="15.355777ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:24:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:24:37.185222334 +0000 UTC m=+153.239756921" watchObservedRunningTime="2026-07-13 21:24:37.187372464 +0000 UTC m=+153.241907436" Jul 13 21:24:39.119992 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:39.119956 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:39.174175 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:39.174145 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:43.523034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:43.522980 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:43.523556 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:43.523065 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:43.527681 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:43.527657 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:44.188780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:44.188751 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:24:44.343695 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:44.343647 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:44.444377 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:44.444289 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:44.512673 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:44.512638 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:46.132079 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:46.132036 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:24:46.236360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:24:46.236329 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:11.608747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:11.608709 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:11.717344 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:11.717309 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:11.992438 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:11.992358 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:12.073407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.073367 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:12.447203 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.447171 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr"] Jul 13 21:25:12.451741 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.451723 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.454345 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.454316 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:25:12.455027 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.455005 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:25:12.455147 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.455049 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccms9\"" Jul 13 21:25:12.459419 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.459394 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr"] Jul 13 21:25:12.584850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.584813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9snb5\" (UniqueName: \"kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.584850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.584856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.585068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.584879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.685613 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.685584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9snb5\" (UniqueName: \"kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.685981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.685624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.685981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.685650 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.685981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.685960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.685981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.685964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.693507 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.693483 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9snb5\" (UniqueName: \"kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.761558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.761472 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:12.889950 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:12.889926 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr"] Jul 13 21:25:12.892526 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:12.892496 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73127cd1_3e80_4778_9864_6446a26c8b02.slice/crio-6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6 WatchSource:0}: Error finding container 6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6: Status 404 returned error can't find the container with id 6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6 Jul 13 21:25:13.285929 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:13.285887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" event={"ID":"73127cd1-3e80-4778-9864-6446a26c8b02","Type":"ContainerStarted","Data":"6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6"} Jul 13 21:25:20.641805 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:20.641759 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:22.321699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:22.321661 2564 generic.go:356] "Generic (PLEG): container finished" podID="73127cd1-3e80-4778-9864-6446a26c8b02" containerID="2ff20e5b7766de8a702d5a9ca82349c25ad35fa54ec512510c7647f6e868f126" exitCode=0 Jul 13 21:25:22.322082 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:22.321734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" event={"ID":"73127cd1-3e80-4778-9864-6446a26c8b02","Type":"ContainerDied","Data":"2ff20e5b7766de8a702d5a9ca82349c25ad35fa54ec512510c7647f6e868f126"} Jul 13 21:25:25.336003 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:25.335965 2564 generic.go:356] "Generic (PLEG): container finished" podID="73127cd1-3e80-4778-9864-6446a26c8b02" containerID="273493d393ef9e9602513f6784f8575d367753e15860c748a7523dc7e4c45f3f" exitCode=0 Jul 13 21:25:25.336377 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:25.336021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" event={"ID":"73127cd1-3e80-4778-9864-6446a26c8b02","Type":"ContainerDied","Data":"273493d393ef9e9602513f6784f8575d367753e15860c748a7523dc7e4c45f3f"} Jul 13 21:25:35.381472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:35.381435 2564 generic.go:356] "Generic (PLEG): container finished" podID="73127cd1-3e80-4778-9864-6446a26c8b02" containerID="4d6aecabd3482db72b24cc403a47c32c9282de37563f02f9c198766b3bcc5b57" exitCode=0 Jul 13 21:25:35.381871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:35.381489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" event={"ID":"73127cd1-3e80-4778-9864-6446a26c8b02","Type":"ContainerDied","Data":"4d6aecabd3482db72b24cc403a47c32c9282de37563f02f9c198766b3bcc5b57"} Jul 13 21:25:36.535433 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.535401 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:36.594760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.594724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9snb5\" (UniqueName: \"kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5\") pod \"73127cd1-3e80-4778-9864-6446a26c8b02\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " Jul 13 21:25:36.594941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.594782 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle\") pod \"73127cd1-3e80-4778-9864-6446a26c8b02\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " Jul 13 21:25:36.594941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.594808 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util\") pod \"73127cd1-3e80-4778-9864-6446a26c8b02\" (UID: \"73127cd1-3e80-4778-9864-6446a26c8b02\") " Jul 13 21:25:36.595504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.595471 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle" (OuterVolumeSpecName: "bundle") pod "73127cd1-3e80-4778-9864-6446a26c8b02" (UID: "73127cd1-3e80-4778-9864-6446a26c8b02"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:25:36.596964 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.596943 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5" (OuterVolumeSpecName: "kube-api-access-9snb5") pod "73127cd1-3e80-4778-9864-6446a26c8b02" (UID: "73127cd1-3e80-4778-9864-6446a26c8b02"). InnerVolumeSpecName "kube-api-access-9snb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:25:36.600560 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.600532 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util" (OuterVolumeSpecName: "util") pod "73127cd1-3e80-4778-9864-6446a26c8b02" (UID: "73127cd1-3e80-4778-9864-6446a26c8b02"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:25:36.695477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.695393 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:36.695477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.695423 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73127cd1-3e80-4778-9864-6446a26c8b02-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:36.695477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:36.695432 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9snb5\" (UniqueName: \"kubernetes.io/projected/73127cd1-3e80-4778-9864-6446a26c8b02-kube-api-access-9snb5\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:37.390771 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:37.390736 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" event={"ID":"73127cd1-3e80-4778-9864-6446a26c8b02","Type":"ContainerDied","Data":"6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6"} Jul 13 21:25:37.391061 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:37.390794 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cf94afa83d866af6dd6995035d4c8af94582444f6344576f77c7b4c6bfeb3a6" Jul 13 21:25:37.391061 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:37.390750 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409t5rr" Jul 13 21:25:38.505065 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:38.505035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:39.447806 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:39.447772 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:40.447202 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447168 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5"] Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447434 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="extract" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447444 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="extract" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447459 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="util" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447464 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="util" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447474 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="pull" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447480 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="pull" Jul 13 21:25:40.447581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.447532 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="73127cd1-3e80-4778-9864-6446a26c8b02" containerName="extract" Jul 13 21:25:40.450684 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.450668 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.453589 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.453561 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 21:25:40.453729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.453705 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 21:25:40.453972 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.453955 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c2ksp\"" Jul 13 21:25:40.471108 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.471087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5"] Jul 13 21:25:40.522937 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.522906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51ab279-1381-41b5-9f49-e0242bf3d343-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.523099 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.522956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9qwm\" (UniqueName: \"kubernetes.io/projected/a51ab279-1381-41b5-9f49-e0242bf3d343-kube-api-access-t9qwm\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.624112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.624059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51ab279-1381-41b5-9f49-e0242bf3d343-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.624298 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.624223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9qwm\" (UniqueName: \"kubernetes.io/projected/a51ab279-1381-41b5-9f49-e0242bf3d343-kube-api-access-t9qwm\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.624476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.624457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51ab279-1381-41b5-9f49-e0242bf3d343-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.631711 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.631686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9qwm\" (UniqueName: \"kubernetes.io/projected/a51ab279-1381-41b5-9f49-e0242bf3d343-kube-api-access-t9qwm\") pod \"cert-manager-operator-controller-manager-55c444bf86-x27g5\" (UID: \"a51ab279-1381-41b5-9f49-e0242bf3d343\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.759655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.759564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" Jul 13 21:25:40.919220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:40.919185 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5"] Jul 13 21:25:40.922426 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:40.922402 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51ab279_1381_41b5_9f49_e0242bf3d343.slice/crio-f23de192d9df3708cf500ded328d681de6e1a6e47f2ebe0222211139ff8e99c9 WatchSource:0}: Error finding container f23de192d9df3708cf500ded328d681de6e1a6e47f2ebe0222211139ff8e99c9: Status 404 returned error can't find the container with id f23de192d9df3708cf500ded328d681de6e1a6e47f2ebe0222211139ff8e99c9 Jul 13 21:25:41.407019 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:41.406986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" event={"ID":"a51ab279-1381-41b5-9f49-e0242bf3d343","Type":"ContainerStarted","Data":"f23de192d9df3708cf500ded328d681de6e1a6e47f2ebe0222211139ff8e99c9"} Jul 13 21:25:42.238950 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:42.238895 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:42.343354 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:42.343316 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:43.417855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:43.417817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" event={"ID":"a51ab279-1381-41b5-9f49-e0242bf3d343","Type":"ContainerStarted","Data":"54801ad372bd1cee853f114fb09fe2c841ac69f2f93313e65f837ad16cff9687"} Jul 13 21:25:43.437838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:43.437778 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x27g5" podStartSLOduration=1.436777124 podStartE2EDuration="3.437753331s" totalImagesPullingTime="2.000976207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:25:43.436776057 +0000 UTC m=+219.491310645" watchObservedRunningTime="2026-07-13 21:25:43.437753331 +0000 UTC m=+219.492287919" Jul 13 21:25:43.648512 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:43.648470 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:43.785420 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:43.785384 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:43.910337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:43.910300 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:44.731797 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.731751 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm"] Jul 13 21:25:44.735286 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.735268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.737600 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.737579 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:25:44.737724 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.737659 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:25:44.738521 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.738499 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccms9\"" Jul 13 21:25:44.746022 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.745999 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm"] Jul 13 21:25:44.761506 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.761473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtc8q\" (UniqueName: \"kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.761650 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.761582 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.761650 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.761620 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.862387 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.862353 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.862387 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.862401 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.862655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.862441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtc8q\" (UniqueName: \"kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.862867 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.862842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.862916 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.862842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:44.874992 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:44.874967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtc8q\" (UniqueName: \"kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:45.044236 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:45.044196 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:45.173156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:45.173129 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm"] Jul 13 21:25:45.175326 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:45.175296 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c68c2b_b9a4_4b0a_9546_f889e6c8a2ab.slice/crio-51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a WatchSource:0}: Error finding container 51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a: Status 404 returned error can't find the container with id 51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a Jul 13 21:25:45.426314 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:45.426207 2564 generic.go:356] "Generic (PLEG): container finished" podID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerID="36cbe2b834857af03a761fb924377acf07428e17a0cabadbfbca20273d14c9a9" exitCode=0 Jul 13 21:25:45.426483 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:45.426298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" event={"ID":"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab","Type":"ContainerDied","Data":"36cbe2b834857af03a761fb924377acf07428e17a0cabadbfbca20273d14c9a9"} Jul 13 21:25:45.426483 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:45.426357 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" event={"ID":"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab","Type":"ContainerStarted","Data":"51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a"} Jul 13 21:25:46.037282 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:46.037249 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:46.254154 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:46.254056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:47.432269 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.432247 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j"] Jul 13 21:25:47.436454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.436436 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.438731 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.438712 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2bksc\"" Jul 13 21:25:47.438836 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.438755 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 21:25:47.439595 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.439570 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 21:25:47.441529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.441508 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j"] Jul 13 21:25:47.485099 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.485075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5txb\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-kube-api-access-v5txb\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.485214 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.485134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.585828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.585797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5txb\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-kube-api-access-v5txb\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.585986 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.585841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.594301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.594273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.594431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.594349 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5txb\" (UniqueName: \"kubernetes.io/projected/d02c8d1c-ad78-49a6-b421-5d1f152c9a60-kube-api-access-v5txb\") pod \"cert-manager-cainjector-559cdd8c87-l4s2j\" (UID: \"d02c8d1c-ad78-49a6-b421-5d1f152c9a60\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.778030 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.777994 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" Jul 13 21:25:47.904977 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:47.904948 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j"] Jul 13 21:25:47.907269 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:47.907247 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd02c8d1c_ad78_49a6_b421_5d1f152c9a60.slice/crio-d28a196d10c829dd81c9d42ffbe2fe9455e52a9f64393b97b551f85dd2e3a7ef WatchSource:0}: Error finding container d28a196d10c829dd81c9d42ffbe2fe9455e52a9f64393b97b551f85dd2e3a7ef: Status 404 returned error can't find the container with id d28a196d10c829dd81c9d42ffbe2fe9455e52a9f64393b97b551f85dd2e3a7ef Jul 13 21:25:48.440127 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:48.440075 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" event={"ID":"d02c8d1c-ad78-49a6-b421-5d1f152c9a60","Type":"ContainerStarted","Data":"d28a196d10c829dd81c9d42ffbe2fe9455e52a9f64393b97b551f85dd2e3a7ef"} Jul 13 21:25:48.441853 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:48.441819 2564 generic.go:356] "Generic (PLEG): container finished" podID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerID="cc783ae0824ae79324b30111725a9e7029c514058d92db77bf1812d2b0c1176a" exitCode=0 Jul 13 21:25:48.441981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:48.441870 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" event={"ID":"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab","Type":"ContainerDied","Data":"cc783ae0824ae79324b30111725a9e7029c514058d92db77bf1812d2b0c1176a"} Jul 13 21:25:48.898915 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:48.898870 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:48.963597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:48.963558 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:49.449003 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:49.448971 2564 generic.go:356] "Generic (PLEG): container finished" podID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerID="32179515eab098f7ba7537411b76aa06286e0b75f1c6db834832b9a6e7469beb" exitCode=0 Jul 13 21:25:49.449441 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:49.449012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" event={"ID":"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab","Type":"ContainerDied","Data":"32179515eab098f7ba7537411b76aa06286e0b75f1c6db834832b9a6e7469beb"} Jul 13 21:25:50.645324 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.645297 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:50.717363 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.717285 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle\") pod \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " Jul 13 21:25:50.717363 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.717324 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtc8q\" (UniqueName: \"kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q\") pod \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " Jul 13 21:25:50.717363 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.717343 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util\") pod \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\" (UID: \"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab\") " Jul 13 21:25:50.717735 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.717706 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle" (OuterVolumeSpecName: "bundle") pod "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" (UID: "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:25:50.719418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.719388 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q" (OuterVolumeSpecName: "kube-api-access-qtc8q") pod "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" (UID: "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab"). InnerVolumeSpecName "kube-api-access-qtc8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:25:50.723024 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.722997 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util" (OuterVolumeSpecName: "util") pod "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" (UID: "b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:25:50.817896 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.817863 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:50.817896 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.817896 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtc8q\" (UniqueName: \"kubernetes.io/projected/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-kube-api-access-qtc8q\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:50.818095 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:50.817908 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:25:51.459235 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:51.459198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" event={"ID":"d02c8d1c-ad78-49a6-b421-5d1f152c9a60","Type":"ContainerStarted","Data":"e5c151b8a3d0b39fb5e2f7bfc39e7d516fb3289382d5dcabdf6f605afe1565aa"} Jul 13 21:25:51.461014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:51.460986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" event={"ID":"b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab","Type":"ContainerDied","Data":"51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a"} Jul 13 21:25:51.461157 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:51.461023 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51f3ce828d915c0ed538743beecdf70f9e2575cc39240dae5836e856ae36de5a" Jul 13 21:25:51.461157 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:51.461022 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbszxm" Jul 13 21:25:51.476812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:51.476767 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-l4s2j" podStartSLOduration=1.813183627 podStartE2EDuration="4.47675031s" totalImagesPullingTime="2.663566683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:25:51.47579032 +0000 UTC m=+227.530324908" watchObservedRunningTime="2026-07-13 21:25:51.47675031 +0000 UTC m=+227.531284897" Jul 13 21:25:52.035468 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035434 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wj9sk"] Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035722 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="extract" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035732 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="extract" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035743 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="util" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035748 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="util" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035758 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="pull" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035763 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="pull" Jul 13 21:25:52.035838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.035802 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c68c2b-b9a4-4b0a-9546-f889e6c8a2ab" containerName="extract" Jul 13 21:25:52.040212 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.040194 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.042452 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.042434 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-nc2c7\"" Jul 13 21:25:52.048063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.048043 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wj9sk"] Jul 13 21:25:52.130941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.130906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlgcl\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-kube-api-access-tlgcl\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.131106 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.130958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.231528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.231487 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgcl\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-kube-api-access-tlgcl\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.231734 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.231539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.240578 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.240552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.240704 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.240685 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgcl\" (UniqueName: \"kubernetes.io/projected/f9feeaf5-c71a-46a1-9b2b-87083517ddb6-kube-api-access-tlgcl\") pod \"cert-manager-webhook-6486ffd997-wj9sk\" (UID: \"f9feeaf5-c71a-46a1-9b2b-87083517ddb6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.349678 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.349584 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:52.475295 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.475274 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wj9sk"] Jul 13 21:25:52.477588 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:52.477561 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9feeaf5_c71a_46a1_9b2b_87083517ddb6.slice/crio-1b033249ea8b112faa78236d25cb6f3f8fa166eb7c6bf34ea89e7f1e5bfc6d5d WatchSource:0}: Error finding container 1b033249ea8b112faa78236d25cb6f3f8fa166eb7c6bf34ea89e7f1e5bfc6d5d: Status 404 returned error can't find the container with id 1b033249ea8b112faa78236d25cb6f3f8fa166eb7c6bf34ea89e7f1e5bfc6d5d Jul 13 21:25:52.726488 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.726410 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:52.832008 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:52.831970 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:53.471036 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:53.471000 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" event={"ID":"f9feeaf5-c71a-46a1-9b2b-87083517ddb6","Type":"ContainerStarted","Data":"384a79702f2621a426c5e33b74b2ff7bc94a891b950651a69f58810e9eb33da3"} Jul 13 21:25:53.471432 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:53.471044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" event={"ID":"f9feeaf5-c71a-46a1-9b2b-87083517ddb6","Type":"ContainerStarted","Data":"1b033249ea8b112faa78236d25cb6f3f8fa166eb7c6bf34ea89e7f1e5bfc6d5d"} Jul 13 21:25:53.471432 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:53.471085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:25:53.487765 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:53.487721 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" podStartSLOduration=1.487705984 podStartE2EDuration="1.487705984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:25:53.486140191 +0000 UTC m=+229.540674775" watchObservedRunningTime="2026-07-13 21:25:53.487705984 +0000 UTC m=+229.542240570" Jul 13 21:25:53.866983 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:53.866946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:54.205779 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.205699 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lj8rr"] Jul 13 21:25:54.210089 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.210072 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.212369 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.212351 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-b7jvg\"" Jul 13 21:25:54.219186 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.219163 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lj8rr"] Jul 13 21:25:54.250476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.250443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-bound-sa-token\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.250476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.250478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44krp\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-kube-api-access-44krp\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.351297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.351258 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-bound-sa-token\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.351297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.351300 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44krp\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-kube-api-access-44krp\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.359135 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.359093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-bound-sa-token\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.359273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.359255 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44krp\" (UniqueName: \"kubernetes.io/projected/b6b92c60-f021-4044-bdeb-a60a90861a23-kube-api-access-44krp\") pod \"cert-manager-55b4d954d5-lj8rr\" (UID: \"b6b92c60-f021-4044-bdeb-a60a90861a23\") " pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.519829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.519799 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lj8rr" Jul 13 21:25:54.651472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:54.651417 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lj8rr"] Jul 13 21:25:54.653542 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:25:54.653513 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6b92c60_f021_4044_bdeb_a60a90861a23.slice/crio-6d28762b9c4b18c1dd60d16ff99c061a077e671d0640846e615ae977a975710b WatchSource:0}: Error finding container 6d28762b9c4b18c1dd60d16ff99c061a077e671d0640846e615ae977a975710b: Status 404 returned error can't find the container with id 6d28762b9c4b18c1dd60d16ff99c061a077e671d0640846e615ae977a975710b Jul 13 21:25:55.312032 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:55.311999 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:55.412552 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:55.412523 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:25:55.481133 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:55.481093 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lj8rr" event={"ID":"b6b92c60-f021-4044-bdeb-a60a90861a23","Type":"ContainerStarted","Data":"a9112233c5e7f7ae64e47a5e7d6a64488f292a6ce89cd4cb23cce6e677314eb5"} Jul 13 21:25:55.481265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:55.481144 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lj8rr" event={"ID":"b6b92c60-f021-4044-bdeb-a60a90861a23","Type":"ContainerStarted","Data":"6d28762b9c4b18c1dd60d16ff99c061a077e671d0640846e615ae977a975710b"} Jul 13 21:25:55.496341 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:55.496297 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lj8rr" podStartSLOduration=1.496280383 podStartE2EDuration="1.496280383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:25:55.49498118 +0000 UTC m=+231.549515767" watchObservedRunningTime="2026-07-13 21:25:55.496280383 +0000 UTC m=+231.550814971" Jul 13 21:25:59.477569 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:25:59.477542 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wj9sk" Jul 13 21:26:01.985878 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:01.985844 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:02.087848 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:02.087812 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:03.779921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.779885 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp"] Jul 13 21:26:03.829501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.829469 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp"] Jul 13 21:26:03.829663 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.829598 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:03.832428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.832405 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 21:26:03.832554 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.832449 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 21:26:03.833266 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.833250 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccms9\"" Jul 13 21:26:03.935642 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.935603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7m8l\" (UniqueName: \"kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:03.935810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.935666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:03.935810 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:03.935758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.018722 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.018681 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:04.036771 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.036699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.036771 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.036753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.036922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.036787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7m8l\" (UniqueName: \"kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.037099 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.037081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.037161 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.037106 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.044588 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.044567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7m8l\" (UniqueName: \"kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.139140 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.139094 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:04.291260 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.291236 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp"] Jul 13 21:26:04.293822 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:04.293796 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12986397_c011_426c_9e06_8dbc47ec9596.slice/crio-a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af WatchSource:0}: Error finding container a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af: Status 404 returned error can't find the container with id a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af Jul 13 21:26:04.521088 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.521050 2564 generic.go:356] "Generic (PLEG): container finished" podID="12986397-c011-426c-9e06-8dbc47ec9596" containerID="7a4ef245011e99ceb3b8d9ae62fc6538562a2b1b9f5267225d163de94495c7ad" exitCode=0 Jul 13 21:26:04.521279 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.521157 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" event={"ID":"12986397-c011-426c-9e06-8dbc47ec9596","Type":"ContainerDied","Data":"7a4ef245011e99ceb3b8d9ae62fc6538562a2b1b9f5267225d163de94495c7ad"} Jul 13 21:26:04.521279 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:04.521218 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" event={"ID":"12986397-c011-426c-9e06-8dbc47ec9596","Type":"ContainerStarted","Data":"a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af"} Jul 13 21:26:05.527344 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:05.527315 2564 generic.go:356] "Generic (PLEG): container finished" podID="12986397-c011-426c-9e06-8dbc47ec9596" containerID="07af776d8caf1d53ff8b5a76a25f2ccf84ec7ebeb7be079af7a14f299c2b67bf" exitCode=0 Jul 13 21:26:05.527749 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:05.527401 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" event={"ID":"12986397-c011-426c-9e06-8dbc47ec9596","Type":"ContainerDied","Data":"07af776d8caf1d53ff8b5a76a25f2ccf84ec7ebeb7be079af7a14f299c2b67bf"} Jul 13 21:26:06.533177 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:06.533141 2564 generic.go:356] "Generic (PLEG): container finished" podID="12986397-c011-426c-9e06-8dbc47ec9596" containerID="e7bdaa3904b9f481f59f1ab497ea7b3c5e98452b89fce8f5f071bab375b216fa" exitCode=0 Jul 13 21:26:06.533542 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:06.533200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" event={"ID":"12986397-c011-426c-9e06-8dbc47ec9596","Type":"ContainerDied","Data":"e7bdaa3904b9f481f59f1ab497ea7b3c5e98452b89fce8f5f071bab375b216fa"} Jul 13 21:26:07.670037 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.670015 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:07.766290 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.766261 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7m8l\" (UniqueName: \"kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l\") pod \"12986397-c011-426c-9e06-8dbc47ec9596\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " Jul 13 21:26:07.766454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.766302 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util\") pod \"12986397-c011-426c-9e06-8dbc47ec9596\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " Jul 13 21:26:07.766454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.766346 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle\") pod \"12986397-c011-426c-9e06-8dbc47ec9596\" (UID: \"12986397-c011-426c-9e06-8dbc47ec9596\") " Jul 13 21:26:07.767073 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.767044 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle" (OuterVolumeSpecName: "bundle") pod "12986397-c011-426c-9e06-8dbc47ec9596" (UID: "12986397-c011-426c-9e06-8dbc47ec9596"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:26:07.768555 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.768530 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l" (OuterVolumeSpecName: "kube-api-access-f7m8l") pod "12986397-c011-426c-9e06-8dbc47ec9596" (UID: "12986397-c011-426c-9e06-8dbc47ec9596"). InnerVolumeSpecName "kube-api-access-f7m8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:26:07.771922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.771894 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util" (OuterVolumeSpecName: "util") pod "12986397-c011-426c-9e06-8dbc47ec9596" (UID: "12986397-c011-426c-9e06-8dbc47ec9596"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:26:07.867601 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.867498 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7m8l\" (UniqueName: \"kubernetes.io/projected/12986397-c011-426c-9e06-8dbc47ec9596-kube-api-access-f7m8l\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:07.867601 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.867549 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:07.867601 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:07.867560 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12986397-c011-426c-9e06-8dbc47ec9596-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:08.281954 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:08.281928 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:08.337380 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:08.337347 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:08.543399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:08.543365 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" event={"ID":"12986397-c011-426c-9e06-8dbc47ec9596","Type":"ContainerDied","Data":"a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af"} Jul 13 21:26:08.543399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:08.543395 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcfmpp" Jul 13 21:26:08.543569 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:08.543413 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a81ac925b73a6bf18f6f1bba0235ca3c09f607375a017e423074390fed9749af" Jul 13 21:26:15.718489 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:15.718452 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:15.817426 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:15.817394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.333573 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.333544 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.556863 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.556827 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.659529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.659446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.679647 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.679615 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6"] Jul 13 21:26:16.680020 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680002 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="pull" Jul 13 21:26:16.680152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680022 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="pull" Jul 13 21:26:16.680152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680038 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="extract" Jul 13 21:26:16.680152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680045 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="extract" Jul 13 21:26:16.680152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680064 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="util" Jul 13 21:26:16.680152 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680072 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="util" Jul 13 21:26:16.680393 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.680184 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12986397-c011-426c-9e06-8dbc47ec9596" containerName="extract" Jul 13 21:26:16.684665 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.684635 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.687495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.687446 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 21:26:16.687495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.687495 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 21:26:16.687728 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.687639 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lnw2r\"" Jul 13 21:26:16.687786 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.687728 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 21:26:16.687836 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.687788 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 21:26:16.699090 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.699065 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6"] Jul 13 21:26:16.845594 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.845550 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.845594 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.845596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.846060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.845698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns25k\" (UniqueName: \"kubernetes.io/projected/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-kube-api-access-ns25k\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.846060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.845748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-manifests\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.846060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.845775 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-tmp\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.856867 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.856833 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.947245 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.947245 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.947245 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947228 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns25k\" (UniqueName: \"kubernetes.io/projected/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-kube-api-access-ns25k\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.947528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-manifests\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.947528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-tmp\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.947745 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.947716 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-manifests\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.949671 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.949642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-tmp\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.949797 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.949781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.949872 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.949851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.954947 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.954923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns25k\" (UniqueName: \"kubernetes.io/projected/ba7cdbc2-36bc-4ea8-81de-b7ede8d24548-kube-api-access-ns25k\") pod \"opendatahub-operator-controller-manager-6f98b9f4-z4rb6\" (UID: \"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548\") " pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:16.955100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.955058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:16.996489 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:16.996454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:17.061368 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.061231 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:17.155437 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.155316 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6"] Jul 13 21:26:17.155795 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.155772 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:17.157771 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:17.157740 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7cdbc2_36bc_4ea8_81de_b7ede8d24548.slice/crio-1852e9bdacdd10fcfd39c9782c72dae43acde0e4cbc900d87b817b8688cdb165 WatchSource:0}: Error finding container 1852e9bdacdd10fcfd39c9782c72dae43acde0e4cbc900d87b817b8688cdb165: Status 404 returned error can't find the container with id 1852e9bdacdd10fcfd39c9782c72dae43acde0e4cbc900d87b817b8688cdb165 Jul 13 21:26:17.231217 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.231139 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:17.329583 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.329550 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:17.583950 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:17.583917 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" event={"ID":"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548","Type":"ContainerStarted","Data":"1852e9bdacdd10fcfd39c9782c72dae43acde0e4cbc900d87b817b8688cdb165"} Jul 13 21:26:20.354024 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:20.353985 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:20.460635 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:20.460556 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:20.602226 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:20.602188 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" event={"ID":"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548","Type":"ContainerStarted","Data":"ec383bc009618a790923d7fbfe5775477d6e5ac19661098c9a5b76e80a2e3663"} Jul 13 21:26:21.608008 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:21.607910 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba7cdbc2-36bc-4ea8-81de-b7ede8d24548" containerID="ec383bc009618a790923d7fbfe5775477d6e5ac19661098c9a5b76e80a2e3663" exitCode=0 Jul 13 21:26:21.608478 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:21.608006 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" event={"ID":"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548","Type":"ContainerDied","Data":"ec383bc009618a790923d7fbfe5775477d6e5ac19661098c9a5b76e80a2e3663"} Jul 13 21:26:22.613941 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:22.613901 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" event={"ID":"ba7cdbc2-36bc-4ea8-81de-b7ede8d24548","Type":"ContainerStarted","Data":"75264adc393e31e91d6b5aceca23b5d63fffb19d7cd742a4de75fb4a11854e48"} Jul 13 21:26:22.614329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:22.614043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:22.636225 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:22.636167 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" podStartSLOduration=2.207722153 podStartE2EDuration="6.636146134s" totalImagesPullingTime="3.327192275s" totalInitContainerRuntime="1.101231706s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:22.633470845 +0000 UTC m=+258.688005433" watchObservedRunningTime="2026-07-13 21:26:22.636146134 +0000 UTC m=+258.690680718" Jul 13 21:26:33.621002 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:33.620919 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f98b9f4-z4rb6" Jul 13 21:26:36.011581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.011548 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:36.136832 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.136794 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:36.680159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.680129 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k7dmc"] Jul 13 21:26:36.684932 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.684913 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.687470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.687451 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 13 21:26:36.687598 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.687474 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k2vv9\"" Jul 13 21:26:36.703343 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.703318 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k7dmc"] Jul 13 21:26:36.707030 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.707010 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:36.822468 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.822432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l5zb\" (UniqueName: \"kubernetes.io/projected/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-kube-api-access-8l5zb\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.822644 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.822481 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-cert\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.827455 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.827413 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:36.923413 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.923375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l5zb\" (UniqueName: \"kubernetes.io/projected/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-kube-api-access-8l5zb\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.923608 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.923443 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-cert\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.926146 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.926102 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-cert\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.931743 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.931669 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l5zb\" (UniqueName: \"kubernetes.io/projected/cfbe5ee8-0fac-4087-ba7f-0305ccffdc25-kube-api-access-8l5zb\") pod \"odh-model-controller-7998c67dc4-k7dmc\" (UID: \"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25\") " pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:36.952009 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.951982 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:36.994836 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:36.994808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:37.050296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.050271 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:37.138422 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.138386 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-k7dmc"] Jul 13 21:26:37.140541 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:37.140511 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbe5ee8_0fac_4087_ba7f_0305ccffdc25.slice/crio-8a3649755ceba11073e85905c2f4c6fb8c539f66e1fcc0e0953f22c9d7c1bb36 WatchSource:0}: Error finding container 8a3649755ceba11073e85905c2f4c6fb8c539f66e1fcc0e0953f22c9d7c1bb36: Status 404 returned error can't find the container with id 8a3649755ceba11073e85905c2f4c6fb8c539f66e1fcc0e0953f22c9d7c1bb36 Jul 13 21:26:37.311889 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.311850 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-74smk"] Jul 13 21:26:37.316405 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.316389 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.318689 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.318664 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 21:26:37.326616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.326586 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:37.327186 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.327165 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-74smk"] Jul 13 21:26:37.427407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.427376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.427407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.427414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrnq\" (UniqueName: \"kubernetes.io/projected/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-kube-api-access-qhrnq\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.527880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.527841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.527880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.527886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrnq\" (UniqueName: \"kubernetes.io/projected/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-kube-api-access-qhrnq\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.528113 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:26:37.527985 2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 21:26:37.528113 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:26:37.528046 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs podName:aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6 nodeName:}" failed. No retries permitted until 2026-07-13 21:26:38.028030323 +0000 UTC m=+274.082564888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs") pod "model-serving-api-54d89bd5fc-74smk" (UID: "aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6") : secret "model-serving-api-tls" not found Jul 13 21:26:37.536529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.536505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrnq\" (UniqueName: \"kubernetes.io/projected/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-kube-api-access-qhrnq\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:37.663596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.663510 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:37.690397 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:37.690363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" event={"ID":"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25","Type":"ContainerStarted","Data":"8a3649755ceba11073e85905c2f4c6fb8c539f66e1fcc0e0953f22c9d7c1bb36"} Jul 13 21:26:38.032000 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:38.031948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:38.035151 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:38.035108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6-tls-certs\") pod \"model-serving-api-54d89bd5fc-74smk\" (UID: \"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6\") " pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:38.227758 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:38.227722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:38.397305 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:38.397280 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-74smk"] Jul 13 21:26:38.399828 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:38.399797 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedaad2f_86a4_4d1d_8b36_ca7b24fe2cf6.slice/crio-a9ce59ec5dee433bb400b91b9f9ecf006e49fa8587a35d165f62efaec97fb4b1 WatchSource:0}: Error finding container a9ce59ec5dee433bb400b91b9f9ecf006e49fa8587a35d165f62efaec97fb4b1: Status 404 returned error can't find the container with id a9ce59ec5dee433bb400b91b9f9ecf006e49fa8587a35d165f62efaec97fb4b1 Jul 13 21:26:38.699310 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:38.699171 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" event={"ID":"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6","Type":"ContainerStarted","Data":"a9ce59ec5dee433bb400b91b9f9ecf006e49fa8587a35d165f62efaec97fb4b1"} Jul 13 21:26:39.301352 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:39.301314 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:39.402470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:39.402427 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:40.711923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:40.711828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" event={"ID":"cfbe5ee8-0fac-4087-ba7f-0305ccffdc25","Type":"ContainerStarted","Data":"ab56a2f1afe73c0d5a30291264ecf426d91d949f52cb990ea4d037f9a1817129"} Jul 13 21:26:40.712452 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:40.711936 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:40.730418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:40.730150 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" podStartSLOduration=1.5244646990000001 podStartE2EDuration="4.730108576s" totalImagesPullingTime="3.205643877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:40.727693139 +0000 UTC m=+276.782227727" watchObservedRunningTime="2026-07-13 21:26:40.730108576 +0000 UTC m=+276.784643164" Jul 13 21:26:41.718292 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.718240 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" event={"ID":"aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6","Type":"ContainerStarted","Data":"5a9e95a152498c280d480b6d92dad4ac793a7c708b9b5e00e0294634ab8d0ef8"} Jul 13 21:26:41.737398 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.737347 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" podStartSLOduration=2.271790217 podStartE2EDuration="4.737330741s" totalImagesPullingTime="2.465540524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:41.736051275 +0000 UTC m=+277.790585861" watchObservedRunningTime="2026-07-13 21:26:41.737330741 +0000 UTC m=+277.791865328" Jul 13 21:26:41.848893 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.848847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:41.905727 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.905685 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2"] Jul 13 21:26:41.908806 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.908782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:41.911062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.911036 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 21:26:41.911062 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.911035 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 21:26:41.911246 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.911212 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zckn2\"" Jul 13 21:26:41.911312 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.911296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 21:26:41.911383 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.911367 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 21:26:41.919804 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.919772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2"] Jul 13 21:26:41.981329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.981258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xskp\" (UniqueName: \"kubernetes.io/projected/13951b00-017d-482f-b3a9-6f0bc0c4b09d-kube-api-access-8xskp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:41.981329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.981321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tmp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:41.981504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:41.981415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tls-certs\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.082735 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.082696 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xskp\" (UniqueName: \"kubernetes.io/projected/13951b00-017d-482f-b3a9-6f0bc0c4b09d-kube-api-access-8xskp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.082922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.082760 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tmp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.082922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.082806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tls-certs\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.085215 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.085189 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tmp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.085347 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.085329 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13951b00-017d-482f-b3a9-6f0bc0c4b09d-tls-certs\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.091443 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.091418 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xskp\" (UniqueName: \"kubernetes.io/projected/13951b00-017d-482f-b3a9-6f0bc0c4b09d-kube-api-access-8xskp\") pod \"kube-auth-proxy-6964b4f475-lgvc2\" (UID: \"13951b00-017d-482f-b3a9-6f0bc0c4b09d\") " pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.218828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.218774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" Jul 13 21:26:42.361956 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.361927 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2"] Jul 13 21:26:42.364379 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:42.364343 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13951b00_017d_482f_b3a9_6f0bc0c4b09d.slice/crio-66503ac8aae8ea924abcc500d00cd543e7fdd2fc67c51f1e5fbb93b4d1e68ea6 WatchSource:0}: Error finding container 66503ac8aae8ea924abcc500d00cd543e7fdd2fc67c51f1e5fbb93b4d1e68ea6: Status 404 returned error can't find the container with id 66503ac8aae8ea924abcc500d00cd543e7fdd2fc67c51f1e5fbb93b4d1e68ea6 Jul 13 21:26:42.724026 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.723989 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" event={"ID":"13951b00-017d-482f-b3a9-6f0bc0c4b09d","Type":"ContainerStarted","Data":"66503ac8aae8ea924abcc500d00cd543e7fdd2fc67c51f1e5fbb93b4d1e68ea6"} Jul 13 21:26:42.724540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:42.724239 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:43.091844 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.091815 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:43.096341 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.096324 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.098874 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.098854 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 21:26:43.098980 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.098911 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9nn5v\"" Jul 13 21:26:43.099131 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.099104 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 21:26:43.099456 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.099437 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 21:26:43.099664 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.099647 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 21:26:43.106419 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.106394 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:43.195217 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195176 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195217 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195439 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195292 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195439 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195318 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx9pj\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195535 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195450 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195535 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.195628 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.195532 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296240 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296198 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296240 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296285 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296376 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296672 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx9pj\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.296975 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.296929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.299098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.299070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.299352 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.299304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.299593 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.299571 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.304389 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.300286 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.304389 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.300878 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:43.305634 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.305610 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx9pj\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.308903 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.308876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token\") pod \"istiod-openshift-gateway-7574459656-846hv\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.406477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.406370 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:43.591367 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.591325 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:43.592632 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:43.592599 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cf4c50_7628_4305_9de0_49a30234fc71.slice/crio-2e824c95d4dc8d98fc5cf34c72386f8cbed28046780bc62988b186c1cfdf77f5 WatchSource:0}: Error finding container 2e824c95d4dc8d98fc5cf34c72386f8cbed28046780bc62988b186c1cfdf77f5: Status 404 returned error can't find the container with id 2e824c95d4dc8d98fc5cf34c72386f8cbed28046780bc62988b186c1cfdf77f5 Jul 13 21:26:43.731179 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:43.731030 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" event={"ID":"b7cf4c50-7628-4305-9de0-49a30234fc71","Type":"ContainerStarted","Data":"2e824c95d4dc8d98fc5cf34c72386f8cbed28046780bc62988b186c1cfdf77f5"} Jul 13 21:26:44.374023 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:44.373983 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:45.539714 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.539679 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w"] Jul 13 21:26:45.546400 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.546376 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.578998 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.578964 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w"] Jul 13 21:26:45.719556 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e7168b-cb97-4168-8b82-72cc999d3272-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719556 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvfqc\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-kube-api-access-pvfqc\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719780 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719741 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.719947 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.719784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.820931 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.820828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.820931 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.820877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.820949 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e7168b-cb97-4168-8b82-72cc999d3272-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.821014 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.821056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.821083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvfqc\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-kube-api-access-pvfqc\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.821129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.821963 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.821928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.823851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.823827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.823969 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.823836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.824162 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.824113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00e7168b-cb97-4168-8b82-72cc999d3272-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.824513 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.824496 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e7168b-cb97-4168-8b82-72cc999d3272-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.829511 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.829487 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvfqc\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-kube-api-access-pvfqc\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.829858 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.829834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e7168b-cb97-4168-8b82-72cc999d3272-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mc94w\" (UID: \"00e7168b-cb97-4168-8b82-72cc999d3272\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:45.857955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:45.857927 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:46.373393 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.373355 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 21:26:46.373523 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.373423 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 21:26:46.531076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.531043 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w"] Jul 13 21:26:46.532574 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:26:46.532548 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e7168b_cb97_4168_8b82_72cc999d3272.slice/crio-60ee375895915047ed18608200efb3ab00f74dbf76f9ada2a1a999e50d4ab12d WatchSource:0}: Error finding container 60ee375895915047ed18608200efb3ab00f74dbf76f9ada2a1a999e50d4ab12d: Status 404 returned error can't find the container with id 60ee375895915047ed18608200efb3ab00f74dbf76f9ada2a1a999e50d4ab12d Jul 13 21:26:46.535509 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.535471 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 21:26:46.535608 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.535558 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 13 21:26:46.753086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.752946 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" event={"ID":"b7cf4c50-7628-4305-9de0-49a30234fc71","Type":"ContainerStarted","Data":"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6"} Jul 13 21:26:46.753086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.753055 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:46.755478 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.755411 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" event={"ID":"00e7168b-cb97-4168-8b82-72cc999d3272","Type":"ContainerStarted","Data":"97b535a25b0993d36a0f3792cbc01a578f02c4d666335334ae8012ac056e1908"} Jul 13 21:26:46.755478 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.755457 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" event={"ID":"00e7168b-cb97-4168-8b82-72cc999d3272","Type":"ContainerStarted","Data":"60ee375895915047ed18608200efb3ab00f74dbf76f9ada2a1a999e50d4ab12d"} Jul 13 21:26:46.755692 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.755503 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:46.757729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.757704 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" event={"ID":"13951b00-017d-482f-b3a9-6f0bc0c4b09d","Type":"ContainerStarted","Data":"080b754d13b7939a13e6ce5d6f1df98598960b09b3445d86aeb4f6ab3451a45f"} Jul 13 21:26:46.774558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.774492 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" podStartSLOduration=0.996665085 podStartE2EDuration="3.774471237s" totalImagesPullingTime="2.777806152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:46.772960217 +0000 UTC m=+282.827494804" watchObservedRunningTime="2026-07-13 21:26:46.774471237 +0000 UTC m=+282.829005827" Jul 13 21:26:46.797345 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.797285 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" podStartSLOduration=1.797264642 podStartE2EDuration="1.797264642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:46.795869505 +0000 UTC m=+282.850404097" watchObservedRunningTime="2026-07-13 21:26:46.797264642 +0000 UTC m=+282.851799230" Jul 13 21:26:46.816469 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:46.816401 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6964b4f475-lgvc2" podStartSLOduration=1.814752147 podStartE2EDuration="5.816378912s" totalImagesPullingTime="4.001626765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:26:46.815279538 +0000 UTC m=+282.869814141" watchObservedRunningTime="2026-07-13 21:26:46.816378912 +0000 UTC m=+282.870913500" Jul 13 21:26:47.432788 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.432754 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:47.621804 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.621751 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:47.765698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.765666 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-846hv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 21:26:47.766074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.765723 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 21:26:47.766074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.765921 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-mc94w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 21:26:47.766074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:47.765965 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" podUID="00e7168b-cb97-4168-8b82-72cc999d3272" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 21:26:48.018491 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:48.018390 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:48.471876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:48.471795 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:48.694046 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:48.694015 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:48.987480 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:48.987445 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:49.082915 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:49.082865 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:49.661273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:49.661236 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:49.757708 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:49.757676 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:50.764948 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:50.764910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mc94w" Jul 13 21:26:50.765487 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:50.764987 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:50.844211 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:50.844180 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:50.844393 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:50.844372 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerName="discovery" containerID="cri-o://10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6" gracePeriod=30 Jul 13 21:26:51.111008 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.110978 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:51.271001 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.270903 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271001 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.270959 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271043 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271071 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx9pj\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271154 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271217 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token\") pod \"b7cf4c50-7628-4305-9de0-49a30234fc71\" (UID: \"b7cf4c50-7628-4305-9de0-49a30234fc71\") " Jul 13 21:26:51.271492 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271429 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:26:51.271577 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.271540 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-ca-configmap\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.273612 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273578 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:26:51.273612 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273597 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj" (OuterVolumeSpecName: "kube-api-access-wx9pj") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "kube-api-access-wx9pj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:26:51.273774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273679 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:26:51.273774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273706 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts" (OuterVolumeSpecName: "cacerts") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:26:51.273774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273725 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:26:51.273873 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.273776 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs" (OuterVolumeSpecName: "local-certs") pod "b7cf4c50-7628-4305-9de0-49a30234fc71" (UID: "b7cf4c50-7628-4305-9de0-49a30234fc71"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:26:51.372533 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372494 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-csr-dns-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.372533 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372526 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7cf4c50-7628-4305-9de0-49a30234fc71-local-certs\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.372533 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372540 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx9pj\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-kube-api-access-wx9pj\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.372777 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372551 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-kubeconfig\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.372777 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372562 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7cf4c50-7628-4305-9de0-49a30234fc71-cacerts\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.372777 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.372573 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7cf4c50-7628-4305-9de0-49a30234fc71-istio-token\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:26:51.720499 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.720424 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-k7dmc" Jul 13 21:26:51.784168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.784108 2564 generic.go:356] "Generic (PLEG): container finished" podID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerID="10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6" exitCode=0 Jul 13 21:26:51.784168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.784162 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" Jul 13 21:26:51.784168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.784148 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" event={"ID":"b7cf4c50-7628-4305-9de0-49a30234fc71","Type":"ContainerDied","Data":"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6"} Jul 13 21:26:51.784685 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.784215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-846hv" event={"ID":"b7cf4c50-7628-4305-9de0-49a30234fc71","Type":"ContainerDied","Data":"2e824c95d4dc8d98fc5cf34c72386f8cbed28046780bc62988b186c1cfdf77f5"} Jul 13 21:26:51.784685 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.784244 2564 scope.go:117] "RemoveContainer" containerID="10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6" Jul 13 21:26:51.794727 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.794709 2564 scope.go:117] "RemoveContainer" containerID="10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6" Jul 13 21:26:51.794981 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:26:51.794966 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6\": container with ID starting with 10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6 not found: ID does not exist" containerID="10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6" Jul 13 21:26:51.795034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.794989 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6"} err="failed to get container status \"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6\": rpc error: code = NotFound desc = could not find container \"10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6\": container with ID starting with 10a63b5209db1af119fd79402f1f69d15acce204b08c9ee9498d3db8136c02d6 not found: ID does not exist" Jul 13 21:26:51.820741 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.820711 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:51.826264 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:51.826239 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-846hv"] Jul 13 21:26:52.529706 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:52.529670 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" path="/var/lib/kubelet/pods/b7cf4c50-7628-4305-9de0-49a30234fc71/volumes" Jul 13 21:26:53.735541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:53.735514 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-74smk" Jul 13 21:26:58.981581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:58.981549 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:26:59.082923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:26:59.082888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:03.686798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:03.686765 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:03.745269 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:03.745218 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:04.352605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:04.352583 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 21:27:21.226501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:21.226468 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:21.329141 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:21.329080 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:37.375533 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.375488 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:37.477380 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.477341 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:37.485852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.485823 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:37.486607 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.486582 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerName="discovery" Jul 13 21:27:37.486749 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.486739 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerName="discovery" Jul 13 21:27:37.486945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.486933 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7cf4c50-7628-4305-9de0-49a30234fc71" containerName="discovery" Jul 13 21:27:37.490628 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.490608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:37.493220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.493199 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6w2r6\"" Jul 13 21:27:37.493417 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.493226 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 21:27:37.494025 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.494002 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 21:27:37.498647 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.498622 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:37.535274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.535246 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:37.560600 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.560570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtjqz\" (UniqueName: \"kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz\") pod \"kuadrant-operator-catalog-vvr4s\" (UID: \"7bfe7648-c444-4f63-85fb-63f8472ababb\") " pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:37.661590 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.661499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtjqz\" (UniqueName: \"kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz\") pod \"kuadrant-operator-catalog-vvr4s\" (UID: \"7bfe7648-c444-4f63-85fb-63f8472ababb\") " pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:37.669570 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.669542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtjqz\" (UniqueName: \"kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz\") pod \"kuadrant-operator-catalog-vvr4s\" (UID: \"7bfe7648-c444-4f63-85fb-63f8472ababb\") " pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:37.807814 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.807779 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:37.848849 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.848816 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:37.949171 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.949149 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:37.950867 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:37.950843 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfe7648_c444_4f63_85fb_63f8472ababb.slice/crio-b62697d0ca83f48eaa3d5677803e1e3ad227cb583e6e3c40b32cb4653d38304f WatchSource:0}: Error finding container b62697d0ca83f48eaa3d5677803e1e3ad227cb583e6e3c40b32cb4653d38304f: Status 404 returned error can't find the container with id b62697d0ca83f48eaa3d5677803e1e3ad227cb583e6e3c40b32cb4653d38304f Jul 13 21:27:37.952249 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:37.952233 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:27:38.006178 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.006143 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" event={"ID":"7bfe7648-c444-4f63-85fb-63f8472ababb","Type":"ContainerStarted","Data":"b62697d0ca83f48eaa3d5677803e1e3ad227cb583e6e3c40b32cb4653d38304f"} Jul 13 21:27:38.058384 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.058352 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8fd7"] Jul 13 21:27:38.062883 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.062865 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:38.068044 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.068020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8fd7"] Jul 13 21:27:38.166900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.166861 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjtlw\" (UniqueName: \"kubernetes.io/projected/ade546ec-b72a-4b4b-b5b3-33a59ca32a49-kube-api-access-zjtlw\") pod \"kuadrant-operator-catalog-h8fd7\" (UID: \"ade546ec-b72a-4b4b-b5b3-33a59ca32a49\") " pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:38.267541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.267505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjtlw\" (UniqueName: \"kubernetes.io/projected/ade546ec-b72a-4b4b-b5b3-33a59ca32a49-kube-api-access-zjtlw\") pod \"kuadrant-operator-catalog-h8fd7\" (UID: \"ade546ec-b72a-4b4b-b5b3-33a59ca32a49\") " pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:38.276641 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.276612 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjtlw\" (UniqueName: \"kubernetes.io/projected/ade546ec-b72a-4b4b-b5b3-33a59ca32a49-kube-api-access-zjtlw\") pod \"kuadrant-operator-catalog-h8fd7\" (UID: \"ade546ec-b72a-4b4b-b5b3-33a59ca32a49\") " pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:38.289982 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.289949 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:38.373241 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.373199 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:38.508821 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.508785 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:27:38.509575 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:38.509555 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8fd7"] Jul 13 21:27:38.511015 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:38.510991 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade546ec_b72a_4b4b_b5b3_33a59ca32a49.slice/crio-8f8455fe2749bafb8b20abbdcc2ccaee613e87b30ae1c319f556bca42789617e WatchSource:0}: Error finding container 8f8455fe2749bafb8b20abbdcc2ccaee613e87b30ae1c319f556bca42789617e: Status 404 returned error can't find the container with id 8f8455fe2749bafb8b20abbdcc2ccaee613e87b30ae1c319f556bca42789617e Jul 13 21:27:39.012008 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:39.011974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" event={"ID":"ade546ec-b72a-4b4b-b5b3-33a59ca32a49","Type":"ContainerStarted","Data":"8f8455fe2749bafb8b20abbdcc2ccaee613e87b30ae1c319f556bca42789617e"} Jul 13 21:27:41.024518 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.024476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" event={"ID":"7bfe7648-c444-4f63-85fb-63f8472ababb","Type":"ContainerStarted","Data":"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537"} Jul 13 21:27:41.024972 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.024552 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" podUID="7bfe7648-c444-4f63-85fb-63f8472ababb" containerName="registry-server" containerID="cri-o://b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537" gracePeriod=2 Jul 13 21:27:41.026087 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.026059 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" event={"ID":"ade546ec-b72a-4b4b-b5b3-33a59ca32a49","Type":"ContainerStarted","Data":"43497ccccc364e8b209a16f3536cad65944cb3903159b14389cb03c92b6e2e03"} Jul 13 21:27:41.040247 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.040209 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" podStartSLOduration=1.621059193 podStartE2EDuration="4.040193492s" totalImagesPullingTime="2.419134299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:27:41.038546742 +0000 UTC m=+337.093081330" watchObservedRunningTime="2026-07-13 21:27:41.040193492 +0000 UTC m=+337.094728079" Jul 13 21:27:41.053073 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.053034 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" podStartSLOduration=1.191262723 podStartE2EDuration="3.053020637s" totalImagesPullingTime="1.861757914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:27:41.052317367 +0000 UTC m=+337.106851955" watchObservedRunningTime="2026-07-13 21:27:41.053020637 +0000 UTC m=+337.107555223" Jul 13 21:27:41.278921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.278856 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:41.397408 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.397371 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtjqz\" (UniqueName: \"kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz\") pod \"7bfe7648-c444-4f63-85fb-63f8472ababb\" (UID: \"7bfe7648-c444-4f63-85fb-63f8472ababb\") " Jul 13 21:27:41.399619 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.399587 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz" (OuterVolumeSpecName: "kube-api-access-dtjqz") pod "7bfe7648-c444-4f63-85fb-63f8472ababb" (UID: "7bfe7648-c444-4f63-85fb-63f8472ababb"). InnerVolumeSpecName "kube-api-access-dtjqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:27:41.498253 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:41.498213 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtjqz\" (UniqueName: \"kubernetes.io/projected/7bfe7648-c444-4f63-85fb-63f8472ababb-kube-api-access-dtjqz\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:42.034366 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.034328 2564 generic.go:356] "Generic (PLEG): container finished" podID="7bfe7648-c444-4f63-85fb-63f8472ababb" containerID="b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537" exitCode=0 Jul 13 21:27:42.034851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.034403 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" Jul 13 21:27:42.034851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.034414 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" event={"ID":"7bfe7648-c444-4f63-85fb-63f8472ababb","Type":"ContainerDied","Data":"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537"} Jul 13 21:27:42.034851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.034466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vvr4s" event={"ID":"7bfe7648-c444-4f63-85fb-63f8472ababb","Type":"ContainerDied","Data":"b62697d0ca83f48eaa3d5677803e1e3ad227cb583e6e3c40b32cb4653d38304f"} Jul 13 21:27:42.034851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.034484 2564 scope.go:117] "RemoveContainer" containerID="b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537" Jul 13 21:27:42.043333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.043313 2564 scope.go:117] "RemoveContainer" containerID="b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537" Jul 13 21:27:42.043577 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:27:42.043561 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537\": container with ID starting with b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537 not found: ID does not exist" containerID="b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537" Jul 13 21:27:42.043618 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.043586 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537"} err="failed to get container status \"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537\": rpc error: code = NotFound desc = could not find container \"b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537\": container with ID starting with b6f7b5df35285f333c9b79ed8ff3d306ba10918d4f19714606a68f320f994537 not found: ID does not exist" Jul 13 21:27:42.059786 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.059759 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:42.072630 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.072599 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vvr4s"] Jul 13 21:27:42.529031 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:42.528998 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bfe7648-c444-4f63-85fb-63f8472ababb" path="/var/lib/kubelet/pods/7bfe7648-c444-4f63-85fb-63f8472ababb/volumes" Jul 13 21:27:48.375234 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:48.373384 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:48.375234 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:48.375075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:48.402984 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:48.402949 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:49.089901 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:49.089873 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h8fd7" Jul 13 21:27:52.892825 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.892787 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q"] Jul 13 21:27:52.893318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.893112 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfe7648-c444-4f63-85fb-63f8472ababb" containerName="registry-server" Jul 13 21:27:52.893318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.893140 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfe7648-c444-4f63-85fb-63f8472ababb" containerName="registry-server" Jul 13 21:27:52.893318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.893231 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfe7648-c444-4f63-85fb-63f8472ababb" containerName="registry-server" Jul 13 21:27:52.897816 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.897798 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:52.900177 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.900157 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x5mck\"" Jul 13 21:27:52.903524 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.903484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q"] Jul 13 21:27:52.994065 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.994031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:52.994253 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.994090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2hql\" (UniqueName: \"kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:52.994253 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:52.994147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.094658 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.094626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.094820 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.094676 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2hql\" (UniqueName: \"kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.094820 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.094703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.095063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.095048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.095157 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.095047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.102676 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.102646 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2hql\" (UniqueName: \"kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.207690 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.207612 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:53.291285 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.291254 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69"] Jul 13 21:27:53.296031 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.296004 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.302619 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.302591 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69"] Jul 13 21:27:53.345709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.345687 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q"] Jul 13 21:27:53.347411 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:53.347386 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5454d9_64eb_4263_9fd4_38c83560fc85.slice/crio-a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf WatchSource:0}: Error finding container a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf: Status 404 returned error can't find the container with id a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf Jul 13 21:27:53.396352 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.396321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.396491 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.396400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9xr6\" (UniqueName: \"kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.396540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.396522 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.497537 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.497503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.497745 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.497553 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.497745 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.497582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9xr6\" (UniqueName: \"kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.497952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.497927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.498050 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.497968 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.505378 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.505356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9xr6\" (UniqueName: \"kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.608546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.608491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:53.737830 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.737809 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69"] Jul 13 21:27:53.739410 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:53.739384 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f759ab1_627c_4bb2_bc0a_eb7706ea547a.slice/crio-bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307 WatchSource:0}: Error finding container bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307: Status 404 returned error can't find the container with id bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307 Jul 13 21:27:53.896142 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.895839 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t"] Jul 13 21:27:53.901886 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.901849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:53.904720 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:53.904695 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t"] Jul 13 21:27:54.002558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.002474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.002558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.002512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.002737 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.002569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t559w\" (UniqueName: \"kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.091537 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.091500 2564 generic.go:356] "Generic (PLEG): container finished" podID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerID="cc79b2e677f842b452ea3f5104533e04692a031d8ccb4f6d3650a664f4c95ee7" exitCode=0 Jul 13 21:27:54.091708 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.091556 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" event={"ID":"9f759ab1-627c-4bb2-bc0a-eb7706ea547a","Type":"ContainerDied","Data":"cc79b2e677f842b452ea3f5104533e04692a031d8ccb4f6d3650a664f4c95ee7"} Jul 13 21:27:54.091708 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.091616 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" event={"ID":"9f759ab1-627c-4bb2-bc0a-eb7706ea547a","Type":"ContainerStarted","Data":"bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307"} Jul 13 21:27:54.093158 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.093108 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerID="ca889e54a6dc5ea025d302dac2a3714f8d48aea4b0b09b964f16172a13a38f94" exitCode=0 Jul 13 21:27:54.093293 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.093188 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" event={"ID":"7b5454d9-64eb-4263-9fd4-38c83560fc85","Type":"ContainerDied","Data":"ca889e54a6dc5ea025d302dac2a3714f8d48aea4b0b09b964f16172a13a38f94"} Jul 13 21:27:54.093293 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.093236 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" event={"ID":"7b5454d9-64eb-4263-9fd4-38c83560fc85","Type":"ContainerStarted","Data":"a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf"} Jul 13 21:27:54.103985 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.103965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.104076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.103993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.104076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.104053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t559w\" (UniqueName: \"kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.104399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.104379 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.104452 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.104432 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.113180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.113161 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t559w\" (UniqueName: \"kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.212076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.212041 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:54.344463 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.344437 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t"] Jul 13 21:27:54.346158 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:54.346106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efbf1d6_b12b_47df_9ba7_acb8be596f5e.slice/crio-f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b WatchSource:0}: Error finding container f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b: Status 404 returned error can't find the container with id f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b Jul 13 21:27:54.491685 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.491644 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc"] Jul 13 21:27:54.495426 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.495402 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.503566 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.503537 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc"] Jul 13 21:27:54.506545 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.506516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.506669 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.506580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d9c7\" (UniqueName: \"kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.506669 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.506612 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.607618 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.607580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.607798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.607628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d9c7\" (UniqueName: \"kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.607798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.607657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.608072 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.608052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.608106 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.608069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.616326 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.616304 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d9c7\" (UniqueName: \"kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.812654 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.812624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:54.946235 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:54.946208 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc"] Jul 13 21:27:54.947608 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:27:54.947578 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c876446_9a3d_4188_9d32_b0553e0352ac.slice/crio-bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d WatchSource:0}: Error finding container bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d: Status 404 returned error can't find the container with id bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d Jul 13 21:27:55.079454 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:27:55.079408 2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c876446_9a3d_4188_9d32_b0553e0352ac.slice/crio-conmon-7bf2ebdef165010b36b65578eaf9308954403520758a48f7d81d2d8ff9b74e07.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c876446_9a3d_4188_9d32_b0553e0352ac.slice/crio-7bf2ebdef165010b36b65578eaf9308954403520758a48f7d81d2d8ff9b74e07.scope\": RecentStats: unable to find data in memory cache]" Jul 13 21:27:55.102517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.102486 2564 generic.go:356] "Generic (PLEG): container finished" podID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerID="7bf2ebdef165010b36b65578eaf9308954403520758a48f7d81d2d8ff9b74e07" exitCode=0 Jul 13 21:27:55.102698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.102577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" event={"ID":"7c876446-9a3d-4188-9d32-b0553e0352ac","Type":"ContainerDied","Data":"7bf2ebdef165010b36b65578eaf9308954403520758a48f7d81d2d8ff9b74e07"} Jul 13 21:27:55.102698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.102632 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" event={"ID":"7c876446-9a3d-4188-9d32-b0553e0352ac","Type":"ContainerStarted","Data":"bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d"} Jul 13 21:27:55.104347 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.104324 2564 generic.go:356] "Generic (PLEG): container finished" podID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerID="6c1fa66ee34113acbcee1abf6587e4e851589f376a39fa9a30d94ef726fa5fee" exitCode=0 Jul 13 21:27:55.104521 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.104407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerDied","Data":"6c1fa66ee34113acbcee1abf6587e4e851589f376a39fa9a30d94ef726fa5fee"} Jul 13 21:27:55.104521 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.104457 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerStarted","Data":"f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b"} Jul 13 21:27:55.106645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.106493 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerID="089105ce1312f15d0debc648c466846f62dc36ced53fe4d084936803bfadb4f1" exitCode=0 Jul 13 21:27:55.106645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:55.106576 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" event={"ID":"7b5454d9-64eb-4263-9fd4-38c83560fc85","Type":"ContainerDied","Data":"089105ce1312f15d0debc648c466846f62dc36ced53fe4d084936803bfadb4f1"} Jul 13 21:27:56.113407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:56.113370 2564 generic.go:356] "Generic (PLEG): container finished" podID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerID="24a74c444a9a2da9e3612a37e615ed4dea035e09170087c55676df8f74aaad94" exitCode=0 Jul 13 21:27:56.113829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:56.113475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" event={"ID":"9f759ab1-627c-4bb2-bc0a-eb7706ea547a","Type":"ContainerDied","Data":"24a74c444a9a2da9e3612a37e615ed4dea035e09170087c55676df8f74aaad94"} Jul 13 21:27:56.116348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:56.116322 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerStarted","Data":"e56169da9334b2a23bd01cc263ccd52d5f4b5205547405623894bd7fe4dbd91f"} Jul 13 21:27:56.118724 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:56.118700 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerID="dfddb8caf926ee7bec2dc98ff076bc396046dd6cccb2726a79b8e9cb5f13be70" exitCode=0 Jul 13 21:27:56.118809 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:56.118765 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" event={"ID":"7b5454d9-64eb-4263-9fd4-38c83560fc85","Type":"ContainerDied","Data":"dfddb8caf926ee7bec2dc98ff076bc396046dd6cccb2726a79b8e9cb5f13be70"} Jul 13 21:27:57.125996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.125960 2564 generic.go:356] "Generic (PLEG): container finished" podID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerID="04cf9a5001652d6381038949d102e662b3c21743c9bfa131ca522a32fcc7d98f" exitCode=0 Jul 13 21:27:57.126429 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.126053 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" event={"ID":"9f759ab1-627c-4bb2-bc0a-eb7706ea547a","Type":"ContainerDied","Data":"04cf9a5001652d6381038949d102e662b3c21743c9bfa131ca522a32fcc7d98f"} Jul 13 21:27:57.127730 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.127705 2564 generic.go:356] "Generic (PLEG): container finished" podID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerID="2adfb249070d588573c5de641e73927e5154655501967e3ad87c166ec3c2fb22" exitCode=0 Jul 13 21:27:57.127867 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.127770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" event={"ID":"7c876446-9a3d-4188-9d32-b0553e0352ac","Type":"ContainerDied","Data":"2adfb249070d588573c5de641e73927e5154655501967e3ad87c166ec3c2fb22"} Jul 13 21:27:57.129522 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.129496 2564 generic.go:356] "Generic (PLEG): container finished" podID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerID="e56169da9334b2a23bd01cc263ccd52d5f4b5205547405623894bd7fe4dbd91f" exitCode=0 Jul 13 21:27:57.129622 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.129597 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerDied","Data":"e56169da9334b2a23bd01cc263ccd52d5f4b5205547405623894bd7fe4dbd91f"} Jul 13 21:27:57.279778 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.279751 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:57.332919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.332886 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle\") pod \"7b5454d9-64eb-4263-9fd4-38c83560fc85\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " Jul 13 21:27:57.333022 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.332931 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2hql\" (UniqueName: \"kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql\") pod \"7b5454d9-64eb-4263-9fd4-38c83560fc85\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " Jul 13 21:27:57.333022 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.333008 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util\") pod \"7b5454d9-64eb-4263-9fd4-38c83560fc85\" (UID: \"7b5454d9-64eb-4263-9fd4-38c83560fc85\") " Jul 13 21:27:57.333427 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.333400 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle" (OuterVolumeSpecName: "bundle") pod "7b5454d9-64eb-4263-9fd4-38c83560fc85" (UID: "7b5454d9-64eb-4263-9fd4-38c83560fc85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:57.334946 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.334928 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql" (OuterVolumeSpecName: "kube-api-access-c2hql") pod "7b5454d9-64eb-4263-9fd4-38c83560fc85" (UID: "7b5454d9-64eb-4263-9fd4-38c83560fc85"). InnerVolumeSpecName "kube-api-access-c2hql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:27:57.338855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.338835 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util" (OuterVolumeSpecName: "util") pod "7b5454d9-64eb-4263-9fd4-38c83560fc85" (UID: "7b5454d9-64eb-4263-9fd4-38c83560fc85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:57.433945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.433863 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:57.433945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.433892 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b5454d9-64eb-4263-9fd4-38c83560fc85-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:57.433945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:57.433900 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2hql\" (UniqueName: \"kubernetes.io/projected/7b5454d9-64eb-4263-9fd4-38c83560fc85-kube-api-access-c2hql\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:58.137715 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.137679 2564 generic.go:356] "Generic (PLEG): container finished" podID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerID="1b74e32a0bd088715d97c404499d3bd949990f5d457c5be14b680ca27cbf520e" exitCode=0 Jul 13 21:27:58.138159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.137758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" event={"ID":"7c876446-9a3d-4188-9d32-b0553e0352ac","Type":"ContainerDied","Data":"1b74e32a0bd088715d97c404499d3bd949990f5d457c5be14b680ca27cbf520e"} Jul 13 21:27:58.139860 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.139839 2564 generic.go:356] "Generic (PLEG): container finished" podID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerID="5183c972aea15c3ccb8fddd8b8ccc9e0a506dbe7607073c8e99d79cb0fad3a7b" exitCode=0 Jul 13 21:27:58.139968 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.139914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerDied","Data":"5183c972aea15c3ccb8fddd8b8ccc9e0a506dbe7607073c8e99d79cb0fad3a7b"} Jul 13 21:27:58.141907 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.141889 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" Jul 13 21:27:58.141907 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.141894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q" event={"ID":"7b5454d9-64eb-4263-9fd4-38c83560fc85","Type":"ContainerDied","Data":"a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf"} Jul 13 21:27:58.142023 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.141927 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2df77073bfeb1b4e12b46284f5582da948bfcf1dc1b9e30b22b85efeb782edf" Jul 13 21:27:58.271524 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.271500 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:58.340789 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.340750 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9xr6\" (UniqueName: \"kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6\") pod \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " Jul 13 21:27:58.340981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.340818 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util\") pod \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " Jul 13 21:27:58.340981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.340877 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle\") pod \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\" (UID: \"9f759ab1-627c-4bb2-bc0a-eb7706ea547a\") " Jul 13 21:27:58.341434 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.341408 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle" (OuterVolumeSpecName: "bundle") pod "9f759ab1-627c-4bb2-bc0a-eb7706ea547a" (UID: "9f759ab1-627c-4bb2-bc0a-eb7706ea547a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:58.343019 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.342992 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6" (OuterVolumeSpecName: "kube-api-access-q9xr6") pod "9f759ab1-627c-4bb2-bc0a-eb7706ea547a" (UID: "9f759ab1-627c-4bb2-bc0a-eb7706ea547a"). InnerVolumeSpecName "kube-api-access-q9xr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:27:58.346267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.346243 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util" (OuterVolumeSpecName: "util") pod "9f759ab1-627c-4bb2-bc0a-eb7706ea547a" (UID: "9f759ab1-627c-4bb2-bc0a-eb7706ea547a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:58.442250 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.442160 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:58.442250 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.442192 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9xr6\" (UniqueName: \"kubernetes.io/projected/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-kube-api-access-q9xr6\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:58.442250 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:58.442202 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f759ab1-627c-4bb2-bc0a-eb7706ea547a-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.149294 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.149267 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" Jul 13 21:27:59.149658 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.149290 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69" event={"ID":"9f759ab1-627c-4bb2-bc0a-eb7706ea547a","Type":"ContainerDied","Data":"bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307"} Jul 13 21:27:59.149658 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.149338 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bad55e5bd2761cfd0a52de90e85b2be5c4cbade6b41fd8f79b54d48e90235307" Jul 13 21:27:59.300351 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.300326 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:27:59.302429 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.302412 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:27:59.349959 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.349923 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle\") pod \"7c876446-9a3d-4188-9d32-b0553e0352ac\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " Jul 13 21:27:59.350158 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.349977 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle\") pod \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " Jul 13 21:27:59.350158 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350008 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t559w\" (UniqueName: \"kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w\") pod \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " Jul 13 21:27:59.350158 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350089 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util\") pod \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\" (UID: \"8efbf1d6-b12b-47df-9ba7-acb8be596f5e\") " Jul 13 21:27:59.350328 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350176 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util\") pod \"7c876446-9a3d-4188-9d32-b0553e0352ac\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " Jul 13 21:27:59.350328 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350235 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d9c7\" (UniqueName: \"kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7\") pod \"7c876446-9a3d-4188-9d32-b0553e0352ac\" (UID: \"7c876446-9a3d-4188-9d32-b0553e0352ac\") " Jul 13 21:27:59.350480 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350458 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle" (OuterVolumeSpecName: "bundle") pod "7c876446-9a3d-4188-9d32-b0553e0352ac" (UID: "7c876446-9a3d-4188-9d32-b0553e0352ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:59.350616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.350596 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.351101 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.351059 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle" (OuterVolumeSpecName: "bundle") pod "8efbf1d6-b12b-47df-9ba7-acb8be596f5e" (UID: "8efbf1d6-b12b-47df-9ba7-acb8be596f5e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:59.352403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.352383 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w" (OuterVolumeSpecName: "kube-api-access-t559w") pod "8efbf1d6-b12b-47df-9ba7-acb8be596f5e" (UID: "8efbf1d6-b12b-47df-9ba7-acb8be596f5e"). InnerVolumeSpecName "kube-api-access-t559w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:27:59.352488 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.352420 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7" (OuterVolumeSpecName: "kube-api-access-8d9c7") pod "7c876446-9a3d-4188-9d32-b0553e0352ac" (UID: "7c876446-9a3d-4188-9d32-b0553e0352ac"). InnerVolumeSpecName "kube-api-access-8d9c7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:27:59.355292 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.355256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util" (OuterVolumeSpecName: "util") pod "8efbf1d6-b12b-47df-9ba7-acb8be596f5e" (UID: "8efbf1d6-b12b-47df-9ba7-acb8be596f5e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:59.355850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.355831 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util" (OuterVolumeSpecName: "util") pod "7c876446-9a3d-4188-9d32-b0553e0352ac" (UID: "7c876446-9a3d-4188-9d32-b0553e0352ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:27:59.451064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.450962 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.451064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.450997 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c876446-9a3d-4188-9d32-b0553e0352ac-util\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.451064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.451011 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d9c7\" (UniqueName: \"kubernetes.io/projected/7c876446-9a3d-4188-9d32-b0553e0352ac-kube-api-access-8d9c7\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.451064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.451024 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:27:59.451064 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:27:59.451038 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t559w\" (UniqueName: \"kubernetes.io/projected/8efbf1d6-b12b-47df-9ba7-acb8be596f5e-kube-api-access-t559w\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:28:00.156286 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.156258 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" Jul 13 21:28:00.156286 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.156266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc" event={"ID":"7c876446-9a3d-4188-9d32-b0553e0352ac","Type":"ContainerDied","Data":"bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d"} Jul 13 21:28:00.156775 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.156319 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc43ca0df98bca677eca1fa9a43ad7bd025b31341aff6f30fb2a9531b659902d" Jul 13 21:28:00.158098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.158078 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" Jul 13 21:28:00.158098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.158092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t" event={"ID":"8efbf1d6-b12b-47df-9ba7-acb8be596f5e","Type":"ContainerDied","Data":"f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b"} Jul 13 21:28:00.158276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:00.158142 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f64a743efccd08d8aecb5908a6ed82a07a7deb3789b326eee793252e2fe1de6b" Jul 13 21:28:06.812161 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:06.812108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:09.447442 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447409 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f"] Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447747 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="extract" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447758 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="extract" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447774 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="pull" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447779 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="pull" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447786 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="util" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447791 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="util" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447798 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="extract" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447803 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="extract" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447812 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="util" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447817 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="util" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447823 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="util" Jul 13 21:28:09.447827 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447830 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="util" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447839 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="util" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447843 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="util" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447850 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447855 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447865 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447869 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447874 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447879 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447886 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447890 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447894 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447899 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="pull" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447952 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efbf1d6-b12b-47df-9ba7-acb8be596f5e" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447960 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f759ab1-627c-4bb2-bc0a-eb7706ea547a" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447967 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b5454d9-64eb-4263-9fd4-38c83560fc85" containerName="extract" Jul 13 21:28:09.448229 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.447973 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c876446-9a3d-4188-9d32-b0553e0352ac" containerName="extract" Jul 13 21:28:09.452397 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.452378 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:09.454622 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.454600 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lr9fl\"" Jul 13 21:28:09.459195 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.459171 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f"] Jul 13 21:28:09.540298 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.540266 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvh4n\" (UniqueName: \"kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n\") pod \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" (UID: \"892d93d0-3f1f-482e-b295-713cb198653d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:09.641304 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.641272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvh4n\" (UniqueName: \"kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n\") pod \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" (UID: \"892d93d0-3f1f-482e-b295-713cb198653d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:09.650322 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.650290 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvh4n\" (UniqueName: \"kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n\") pod \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" (UID: \"892d93d0-3f1f-482e-b295-713cb198653d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:09.763092 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.763062 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:09.902035 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:09.902007 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f"] Jul 13 21:28:09.903946 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:09.903917 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892d93d0_3f1f_482e_b295_713cb198653d.slice/crio-7759bbaf7279aafa1f50f3393595048b2d710d000d9d0216a74ae81de0cdce27 WatchSource:0}: Error finding container 7759bbaf7279aafa1f50f3393595048b2d710d000d9d0216a74ae81de0cdce27: Status 404 returned error can't find the container with id 7759bbaf7279aafa1f50f3393595048b2d710d000d9d0216a74ae81de0cdce27 Jul 13 21:28:10.210507 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:10.210418 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" event={"ID":"892d93d0-3f1f-482e-b295-713cb198653d","Type":"ContainerStarted","Data":"7759bbaf7279aafa1f50f3393595048b2d710d000d9d0216a74ae81de0cdce27"} Jul 13 21:28:10.407796 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:10.407759 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:10.509421 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:10.509344 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:12.222991 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:12.222948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" event={"ID":"892d93d0-3f1f-482e-b295-713cb198653d","Type":"ContainerStarted","Data":"f2edc237b2573b70f2483cddfb01bf1a922a1499bf4931141ada50d1aff161f0"} Jul 13 21:28:12.223370 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:12.223040 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:12.244480 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:12.244420 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" podStartSLOduration=1.148866376 podStartE2EDuration="3.244398708s" totalImagesPullingTime="2.095532332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:12.24210835 +0000 UTC m=+368.296642938" watchObservedRunningTime="2026-07-13 21:28:12.244398708 +0000 UTC m=+368.298933296" Jul 13 21:28:15.607550 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.607516 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq"] Jul 13 21:28:15.615290 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.615268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.617772 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.617746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fwt47\"" Jul 13 21:28:15.623818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.623790 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq"] Jul 13 21:28:15.699530 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.699496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cg77\" (UniqueName: \"kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.699530 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.699535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.800286 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.800254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cg77\" (UniqueName: \"kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.800474 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.800294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.800645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.800628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.815598 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.815562 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cg77\" (UniqueName: \"kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:15.926168 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:15.926046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:16.002818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:16.002765 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:16.065919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:16.065894 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq"] Jul 13 21:28:16.067597 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:16.067576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ebeb3f_0ffa_4011_845e_187a64d3606a.slice/crio-275c835f9d27b3107cf0a58e8d98130808e485dcf28466c3d13caa67559a83e9 WatchSource:0}: Error finding container 275c835f9d27b3107cf0a58e8d98130808e485dcf28466c3d13caa67559a83e9: Status 404 returned error can't find the container with id 275c835f9d27b3107cf0a58e8d98130808e485dcf28466c3d13caa67559a83e9 Jul 13 21:28:16.244730 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:16.244643 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" event={"ID":"a8ebeb3f-0ffa-4011-845e-187a64d3606a","Type":"ContainerStarted","Data":"275c835f9d27b3107cf0a58e8d98130808e485dcf28466c3d13caa67559a83e9"} Jul 13 21:28:16.627090 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:16.627059 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:20.215649 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:20.215610 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:22.279313 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.279274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" event={"ID":"a8ebeb3f-0ffa-4011-845e-187a64d3606a","Type":"ContainerStarted","Data":"5fed2ceab0d3e6d1ec4bb3019c15336ab74a05c9fa03b4b4591ae3d6e2989079"} Jul 13 21:28:22.279711 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.279464 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:22.300414 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.300362 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" podStartSLOduration=1.267018841 podStartE2EDuration="7.300345388s" totalImagesPullingTime="6.033326547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:22.298904257 +0000 UTC m=+378.353438846" watchObservedRunningTime="2026-07-13 21:28:22.300345388 +0000 UTC m=+378.354880043" Jul 13 21:28:22.402597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.402517 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm"] Jul 13 21:28:22.406243 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.406221 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:22.408432 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.408411 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 13 21:28:22.408549 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.408466 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-22xln\"" Jul 13 21:28:22.414667 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.414642 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm"] Jul 13 21:28:22.563977 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.563934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glxq\" (UniqueName: \"kubernetes.io/projected/f8d00ec0-1415-4a01-a499-1dc97089dfbd-kube-api-access-5glxq\") pod \"dns-operator-controller-manager-864dd78c4c-zv9nm\" (UID: \"f8d00ec0-1415-4a01-a499-1dc97089dfbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:22.664693 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.664596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glxq\" (UniqueName: \"kubernetes.io/projected/f8d00ec0-1415-4a01-a499-1dc97089dfbd-kube-api-access-5glxq\") pod \"dns-operator-controller-manager-864dd78c4c-zv9nm\" (UID: \"f8d00ec0-1415-4a01-a499-1dc97089dfbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:22.676866 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.675987 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glxq\" (UniqueName: \"kubernetes.io/projected/f8d00ec0-1415-4a01-a499-1dc97089dfbd-kube-api-access-5glxq\") pod \"dns-operator-controller-manager-864dd78c4c-zv9nm\" (UID: \"f8d00ec0-1415-4a01-a499-1dc97089dfbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:22.718812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.718617 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:22.856683 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:22.856658 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm"] Jul 13 21:28:22.859034 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:22.859006 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d00ec0_1415_4a01_a499_1dc97089dfbd.slice/crio-a29564622e7b940ac2e810f6d5e60e8551e17c6d06a02fdd2749496686364760 WatchSource:0}: Error finding container a29564622e7b940ac2e810f6d5e60e8551e17c6d06a02fdd2749496686364760: Status 404 returned error can't find the container with id a29564622e7b940ac2e810f6d5e60e8551e17c6d06a02fdd2749496686364760 Jul 13 21:28:23.230460 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:23.230405 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:23.285596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:23.285563 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" event={"ID":"f8d00ec0-1415-4a01-a499-1dc97089dfbd","Type":"ContainerStarted","Data":"a29564622e7b940ac2e810f6d5e60e8551e17c6d06a02fdd2749496686364760"} Jul 13 21:28:24.980273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.980236 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv"] Jul 13 21:28:24.985402 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.985371 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:24.987770 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.987742 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 13 21:28:24.987770 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.987765 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 13 21:28:24.987946 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.987747 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x5mck\"" Jul 13 21:28:24.993461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.993408 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv"] Jul 13 21:28:24.997161 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:24.997112 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:25.087716 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.087693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.087829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.087734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.087829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.087761 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hs45\" (UniqueName: \"kubernetes.io/projected/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-kube-api-access-5hs45\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.188583 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.188545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.188750 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.188599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.188750 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.188644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs45\" (UniqueName: \"kubernetes.io/projected/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-kube-api-access-5hs45\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.188827 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:25.188768 2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 13 21:28:25.188860 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:25.188833 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert podName:a2f6cc87-1f8b-4545-ad84-ab9b0645b23f nodeName:}" failed. No retries permitted until 2026-07-13 21:28:25.688817139 +0000 UTC m=+381.743351708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-7j9qv" (UID: "a2f6cc87-1f8b-4545-ad84-ab9b0645b23f") : secret "plugin-serving-cert" not found Jul 13 21:28:25.189321 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.189303 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.199131 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.199090 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hs45\" (UniqueName: \"kubernetes.io/projected/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-kube-api-access-5hs45\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.295355 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.295305 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:25.298628 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.298599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" event={"ID":"f8d00ec0-1415-4a01-a499-1dc97089dfbd","Type":"ContainerStarted","Data":"83c215c26ec84648042029abf9e10e26accf48ad07f18c1625a2b6d045ced106"} Jul 13 21:28:25.298781 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.298661 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:25.317227 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.317176 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" podStartSLOduration=1.115948224 podStartE2EDuration="3.317160499s" totalImagesPullingTime="2.201212275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:25.316046875 +0000 UTC m=+381.370581456" watchObservedRunningTime="2026-07-13 21:28:25.317160499 +0000 UTC m=+381.371695086" Jul 13 21:28:25.616421 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.616338 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d76b96744-cpfx9"] Jul 13 21:28:25.619905 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.619877 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.629478 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.629453 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d76b96744-cpfx9"] Jul 13 21:28:25.670916 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.670881 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:25.693060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.693033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.695469 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.695446 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2f6cc87-1f8b-4545-ad84-ab9b0645b23f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-7j9qv\" (UID: \"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.731644 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.731615 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:25.793729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-oauth-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.793871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-console-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.793871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h2xm\" (UniqueName: \"kubernetes.io/projected/7098a4db-4ac7-4835-bf90-afee7399ac46-kube-api-access-7h2xm\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.793871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.793871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-trusted-ca-bundle\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.793871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-oauth-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.794068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.793877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-service-ca\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.894838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894739 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-oauth-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.894838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-console-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.894838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894820 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h2xm\" (UniqueName: \"kubernetes.io/projected/7098a4db-4ac7-4835-bf90-afee7399ac46-kube-api-access-7h2xm\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.894838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895215 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-trusted-ca-bundle\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895215 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894876 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-oauth-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895215 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.894908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-service-ca\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895668 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.895646 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-service-ca\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895885 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.895862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-oauth-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895885 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.895876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-trusted-ca-bundle\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.895997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.895862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7098a4db-4ac7-4835-bf90-afee7399ac46-console-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.897364 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.897344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-oauth-config\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.897462 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.897442 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7098a4db-4ac7-4835-bf90-afee7399ac46-console-serving-cert\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.898225 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.898203 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" Jul 13 21:28:25.902380 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.902359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h2xm\" (UniqueName: \"kubernetes.io/projected/7098a4db-4ac7-4835-bf90-afee7399ac46-kube-api-access-7h2xm\") pod \"console-5d76b96744-cpfx9\" (UID: \"7098a4db-4ac7-4835-bf90-afee7399ac46\") " pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:25.930475 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:25.930442 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:26.045606 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.045574 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv"] Jul 13 21:28:26.047035 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:26.047010 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2f6cc87_1f8b_4545_ad84_ab9b0645b23f.slice/crio-1ba48b76c3d8500ccabd7c84b4a4bfc0349bcc6f485576e8c3a7e92b8573a0c2 WatchSource:0}: Error finding container 1ba48b76c3d8500ccabd7c84b4a4bfc0349bcc6f485576e8c3a7e92b8573a0c2: Status 404 returned error can't find the container with id 1ba48b76c3d8500ccabd7c84b4a4bfc0349bcc6f485576e8c3a7e92b8573a0c2 Jul 13 21:28:26.087938 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.087903 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d76b96744-cpfx9"] Jul 13 21:28:26.089706 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:26.089679 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7098a4db_4ac7_4835_bf90_afee7399ac46.slice/crio-bdbe53a0a795b1ef4ed0cb09b9ad544f7315638c8837ecc2a880b8fa206ac3ea WatchSource:0}: Error finding container bdbe53a0a795b1ef4ed0cb09b9ad544f7315638c8837ecc2a880b8fa206ac3ea: Status 404 returned error can't find the container with id bdbe53a0a795b1ef4ed0cb09b9ad544f7315638c8837ecc2a880b8fa206ac3ea Jul 13 21:28:26.308507 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.308469 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" event={"ID":"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f","Type":"ContainerStarted","Data":"1ba48b76c3d8500ccabd7c84b4a4bfc0349bcc6f485576e8c3a7e92b8573a0c2"} Jul 13 21:28:26.310086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.310057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76b96744-cpfx9" event={"ID":"7098a4db-4ac7-4835-bf90-afee7399ac46","Type":"ContainerStarted","Data":"0a3484ed74506fdda5072e0c75908cd5f52172b564cefa871692ac8556d77da8"} Jul 13 21:28:26.310246 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.310111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d76b96744-cpfx9" event={"ID":"7098a4db-4ac7-4835-bf90-afee7399ac46","Type":"ContainerStarted","Data":"bdbe53a0a795b1ef4ed0cb09b9ad544f7315638c8837ecc2a880b8fa206ac3ea"} Jul 13 21:28:26.330235 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:26.330188 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d76b96744-cpfx9" podStartSLOduration=1.33017224 podStartE2EDuration="1.33017224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:26.327783238 +0000 UTC m=+382.382317838" watchObservedRunningTime="2026-07-13 21:28:26.33017224 +0000 UTC m=+382.384706827" Jul 13 21:28:28.409133 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:28.409076 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:28.467110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:28.467047 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:33.288949 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:33.288915 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:35.014718 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.014678 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq"] Jul 13 21:28:35.015201 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.014880 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" containerName="manager" containerID="cri-o://5fed2ceab0d3e6d1ec4bb3019c15336ab74a05c9fa03b4b4591ae3d6e2989079" gracePeriod=2 Jul 13 21:28:35.022618 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.022587 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq"] Jul 13 21:28:35.038747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.038715 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f"] Jul 13 21:28:35.039041 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.039010 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" podUID="892d93d0-3f1f-482e-b295-713cb198653d" containerName="manager" containerID="cri-o://f2edc237b2573b70f2483cddfb01bf1a922a1499bf4931141ada50d1aff161f0" gracePeriod=2 Jul 13 21:28:35.041858 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.041804 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:28:35.043981 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.042246 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:35.043981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.042601 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" containerName="manager" Jul 13 21:28:35.043981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.042624 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" containerName="manager" Jul 13 21:28:35.043981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.042733 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" containerName="manager" Jul 13 21:28:35.044283 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.044241 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:35.047358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.047335 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f"] Jul 13 21:28:35.047494 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.047470 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.050991 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.050826 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:35.068113 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.068062 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:28:35.071723 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.071693 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds"] Jul 13 21:28:35.072226 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.072207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892d93d0-3f1f-482e-b295-713cb198653d" containerName="manager" Jul 13 21:28:35.072226 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.072229 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="892d93d0-3f1f-482e-b295-713cb198653d" containerName="manager" Jul 13 21:28:35.072394 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.072340 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="892d93d0-3f1f-482e-b295-713cb198653d" containerName="manager" Jul 13 21:28:35.077094 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.077070 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:35.081648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.080863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdbtb\" (UniqueName: \"kubernetes.io/projected/c51bf87e-7af5-4b09-8a18-997e2a5d4c78-kube-api-access-kdbtb\") pod \"limitador-operator-controller-manager-859d7b78bd-gxmds\" (UID: \"c51bf87e-7af5-4b09-8a18-997e2a5d4c78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:35.081648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.081025 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.081648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.081066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c864\" (UniqueName: \"kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.088486 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.088446 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:35.090427 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.090386 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:35.095485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.095460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds"] Jul 13 21:28:35.114467 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.114405 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:35.183811 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.183776 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdbtb\" (UniqueName: \"kubernetes.io/projected/c51bf87e-7af5-4b09-8a18-997e2a5d4c78-kube-api-access-kdbtb\") pod \"limitador-operator-controller-manager-859d7b78bd-gxmds\" (UID: \"c51bf87e-7af5-4b09-8a18-997e2a5d4c78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:35.183996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.183830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.184084 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.184052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c864\" (UniqueName: \"kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.184327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.184299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.192874 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.192827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c864\" (UniqueName: \"kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4dw9w\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.193134 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.193100 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdbtb\" (UniqueName: \"kubernetes.io/projected/c51bf87e-7af5-4b09-8a18-997e2a5d4c78-kube-api-access-kdbtb\") pod \"limitador-operator-controller-manager-859d7b78bd-gxmds\" (UID: \"c51bf87e-7af5-4b09-8a18-997e2a5d4c78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:35.425829 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.425748 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:35.433392 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.433368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:35.930935 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.930905 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:35.931280 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.931250 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:35.936326 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:35.936308 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:35.962302 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.962258 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:35.964023 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:35.963991 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:36.074234 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.074196 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:36.299981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.299906 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:28:36.317248 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.316775 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zv9nm" Jul 13 21:28:36.317248 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.317097 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.318832 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.318793 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:36.320652 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.320631 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:28:36.321205 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.321171 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:36.375259 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.375205 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:36.377057 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.377024 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:36.381082 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.381057 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d76b96744-cpfx9" Jul 13 21:28:36.395648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.395613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lzj\" (UniqueName: \"kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.395817 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.395748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.412550 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.412509 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:36.414285 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.414252 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:36.443042 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.443013 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:28:36.445056 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.445022 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:36.468136 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:36.468075 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:36.497467 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.497428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lzj\" (UniqueName: \"kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.497655 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.497481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.497960 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.497935 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.507082 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.507041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lzj\" (UniqueName: \"kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj\") pod \"kuadrant-operator-controller-manager-86878c769f-gb7q7\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.633199 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.633112 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:36.637502 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.637466 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:36.749772 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.749740 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:36.887886 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:36.887797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:37.268490 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:37.268456 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:38.422630 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:38.422584 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:38.523664 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:38.523628 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:39.059260 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:39.059221 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:39.155512 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:39.155469 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:39.565403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:39.565369 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:42.015911 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:42.015865 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:42.086067 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:42.086013 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:47.413795 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:47.413761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:47.514566 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:47.514529 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:48.456616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.456553 2564 generic.go:356] "Generic (PLEG): container finished" podID="892d93d0-3f1f-482e-b295-713cb198653d" containerID="f2edc237b2573b70f2483cddfb01bf1a922a1499bf4931141ada50d1aff161f0" exitCode=0 Jul 13 21:28:48.462979 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.461478 2564 generic.go:356] "Generic (PLEG): container finished" podID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" containerID="5fed2ceab0d3e6d1ec4bb3019c15336ab74a05c9fa03b4b4591ae3d6e2989079" exitCode=0 Jul 13 21:28:48.533631 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.533192 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:48.535623 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:48.535483 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:48.549094 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.549069 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:48.549706 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.549671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:28:48.551003 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:48.550977 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:48.552532 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:48.552502 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:48.619574 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.619549 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume\") pod \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " Jul 13 21:28:48.619703 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.619602 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvh4n\" (UniqueName: \"kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n\") pod \"892d93d0-3f1f-482e-b295-713cb198653d\" (UID: \"892d93d0-3f1f-482e-b295-713cb198653d\") " Jul 13 21:28:48.619703 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.619629 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cg77\" (UniqueName: \"kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77\") pod \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\" (UID: \"a8ebeb3f-0ffa-4011-845e-187a64d3606a\") " Jul 13 21:28:48.620132 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.620086 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8ebeb3f-0ffa-4011-845e-187a64d3606a" (UID: "a8ebeb3f-0ffa-4011-845e-187a64d3606a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:28:48.621658 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.621633 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n" (OuterVolumeSpecName: "kube-api-access-kvh4n") pod "892d93d0-3f1f-482e-b295-713cb198653d" (UID: "892d93d0-3f1f-482e-b295-713cb198653d"). InnerVolumeSpecName "kube-api-access-kvh4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:28:48.622091 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.622071 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77" (OuterVolumeSpecName: "kube-api-access-4cg77") pod "a8ebeb3f-0ffa-4011-845e-187a64d3606a" (UID: "a8ebeb3f-0ffa-4011-845e-187a64d3606a"). InnerVolumeSpecName "kube-api-access-4cg77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:28:48.721098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.721057 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8ebeb3f-0ffa-4011-845e-187a64d3606a-extensions-socket-volume\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:28:48.721098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.721098 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvh4n\" (UniqueName: \"kubernetes.io/projected/892d93d0-3f1f-482e-b295-713cb198653d-kube-api-access-kvh4n\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:28:48.721327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.721113 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cg77\" (UniqueName: \"kubernetes.io/projected/a8ebeb3f-0ffa-4011-845e-187a64d3606a-kube-api-access-4cg77\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:28:48.763128 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.762996 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:28:48.765414 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:48.765352 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99f16b1_d8f5_4b04_912b_0c35a9ea8054.slice/crio-d69bd1a5977dbf5b6c6c0b6409b5e733d812b74aa832245db6fc9b5d7fe65de9 WatchSource:0}: Error finding container d69bd1a5977dbf5b6c6c0b6409b5e733d812b74aa832245db6fc9b5d7fe65de9: Status 404 returned error can't find the container with id d69bd1a5977dbf5b6c6c0b6409b5e733d812b74aa832245db6fc9b5d7fe65de9 Jul 13 21:28:48.765884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:48.765862 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds"] Jul 13 21:28:48.767976 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:28:48.767953 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51bf87e_7af5_4b09_8a18_997e2a5d4c78.slice/crio-7d4fdb4dc6ef8a8a307e6ec75a984478f57879915b67670acb7f0d0e5336e284 WatchSource:0}: Error finding container 7d4fdb4dc6ef8a8a307e6ec75a984478f57879915b67670acb7f0d0e5336e284: Status 404 returned error can't find the container with id 7d4fdb4dc6ef8a8a307e6ec75a984478f57879915b67670acb7f0d0e5336e284 Jul 13 21:28:49.468437 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.468396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" event={"ID":"3563a519-5ca9-48a8-b935-bd55e07ac803","Type":"ContainerStarted","Data":"ad1c6cd46342781a2fd4489f32d629c05d0d2340b07184c2609d2a249fdc5e56"} Jul 13 21:28:49.468883 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.468462 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" event={"ID":"3563a519-5ca9-48a8-b935-bd55e07ac803","Type":"ContainerStarted","Data":"9a645dc8854cf4f64947eaf8fd051739d2e7fe28a2f6bf24a48b2e512ea98f41"} Jul 13 21:28:49.468883 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.468528 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:28:49.469764 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.469749 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:49.469870 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.469750 2564 scope.go:117] "RemoveContainer" containerID="5fed2ceab0d3e6d1ec4bb3019c15336ab74a05c9fa03b4b4591ae3d6e2989079" Jul 13 21:28:49.471699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.471673 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" event={"ID":"c99f16b1-d8f5-4b04-912b-0c35a9ea8054","Type":"ContainerStarted","Data":"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a"} Jul 13 21:28:49.471804 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.471724 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:28:49.471804 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.471747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" event={"ID":"c99f16b1-d8f5-4b04-912b-0c35a9ea8054","Type":"ContainerStarted","Data":"d69bd1a5977dbf5b6c6c0b6409b5e733d812b74aa832245db6fc9b5d7fe65de9"} Jul 13 21:28:49.473501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.473376 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" event={"ID":"c51bf87e-7af5-4b09-8a18-997e2a5d4c78","Type":"ContainerStarted","Data":"db8cb217999862781adba0cf5cbc98593b8d3896b2b013fae5d7b4d1869e9bcd"} Jul 13 21:28:49.473501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.473434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:28:49.473501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.473460 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" event={"ID":"c51bf87e-7af5-4b09-8a18-997e2a5d4c78","Type":"ContainerStarted","Data":"7d4fdb4dc6ef8a8a307e6ec75a984478f57879915b67670acb7f0d0e5336e284"} Jul 13 21:28:49.474993 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.474970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" event={"ID":"a2f6cc87-1f8b-4545-ad84-ab9b0645b23f","Type":"ContainerStarted","Data":"b2f4460492a8463631148c32513895a215984c19cc70f6f535fb6b5c2908245f"} Jul 13 21:28:49.476462 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.476447 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:49.479826 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.479812 2564 scope.go:117] "RemoveContainer" containerID="f2edc237b2573b70f2483cddfb01bf1a922a1499bf4931141ada50d1aff161f0" Jul 13 21:28:49.492056 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:49.492028 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8ghsq\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8ghsq" Jul 13 21:28:49.492633 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.492597 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" podStartSLOduration=13.492582425 podStartE2EDuration="13.492582425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:49.490209782 +0000 UTC m=+405.544744370" watchObservedRunningTime="2026-07-13 21:28:49.492582425 +0000 UTC m=+405.547117011" Jul 13 21:28:49.511999 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.511955 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" podStartSLOduration=14.511940899 podStartE2EDuration="14.511940899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:49.510287839 +0000 UTC m=+405.564822429" watchObservedRunningTime="2026-07-13 21:28:49.511940899 +0000 UTC m=+405.566475485" Jul 13 21:28:49.527770 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:28:49.527736 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2rn4f\" is forbidden: User \"system:node:ip-10-0-131-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-109.ec2.internal' and this object" podUID="892d93d0-3f1f-482e-b295-713cb198653d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rn4f" Jul 13 21:28:49.528059 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.528022 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" podStartSLOduration=14.528010412 podStartE2EDuration="14.528010412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:49.525886331 +0000 UTC m=+405.580420919" watchObservedRunningTime="2026-07-13 21:28:49.528010412 +0000 UTC m=+405.582545000" Jul 13 21:28:49.548692 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:49.548635 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-7j9qv" podStartSLOduration=3.153777838 podStartE2EDuration="25.548611654s" totalImagesPullingTime="22.394833816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:28:49.546028256 +0000 UTC m=+405.600562846" watchObservedRunningTime="2026-07-13 21:28:49.548611654 +0000 UTC m=+405.603146243" Jul 13 21:28:50.528521 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:50.528488 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892d93d0-3f1f-482e-b295-713cb198653d" path="/var/lib/kubelet/pods/892d93d0-3f1f-482e-b295-713cb198653d/volumes" Jul 13 21:28:50.528877 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:50.528854 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8ebeb3f-0ffa-4011-845e-187a64d3606a" path="/var/lib/kubelet/pods/a8ebeb3f-0ffa-4011-845e-187a64d3606a/volumes" Jul 13 21:28:51.220411 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:51.220382 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:51.320681 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:51.320640 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:59.233663 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:59.233611 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:28:59.331000 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:28:59.330968 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:00.486002 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.485964 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:29:00.486397 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.486183 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gxmds" Jul 13 21:29:00.486397 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.486244 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:29:00.578246 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.578216 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:29:00.578448 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.578415 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" podUID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" containerName="manager" containerID="cri-o://37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a" gracePeriod=10 Jul 13 21:29:00.847281 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.847257 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:29:00.930972 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.930934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume\") pod \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " Jul 13 21:29:00.931197 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.930991 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c864\" (UniqueName: \"kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864\") pod \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\" (UID: \"c99f16b1-d8f5-4b04-912b-0c35a9ea8054\") " Jul 13 21:29:00.931410 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.931384 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c99f16b1-d8f5-4b04-912b-0c35a9ea8054" (UID: "c99f16b1-d8f5-4b04-912b-0c35a9ea8054"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:29:00.933074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:00.933048 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864" (OuterVolumeSpecName: "kube-api-access-5c864") pod "c99f16b1-d8f5-4b04-912b-0c35a9ea8054" (UID: "c99f16b1-d8f5-4b04-912b-0c35a9ea8054"). InnerVolumeSpecName "kube-api-access-5c864". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:29:01.032715 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.032680 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-extensions-socket-volume\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.032715 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.032709 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c864\" (UniqueName: \"kubernetes.io/projected/c99f16b1-d8f5-4b04-912b-0c35a9ea8054-kube-api-access-5c864\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.470086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.469969 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-696c6d8b9b-6wchx" podUID="530eecca-2abb-4271-b41c-d00a266daa91" containerName="console" containerID="cri-o://4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247" gracePeriod=15 Jul 13 21:29:01.544374 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.544336 2564 generic.go:356] "Generic (PLEG): container finished" podID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" containerID="37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a" exitCode=0 Jul 13 21:29:01.544754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.544377 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" Jul 13 21:29:01.544754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.544421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" event={"ID":"c99f16b1-d8f5-4b04-912b-0c35a9ea8054","Type":"ContainerDied","Data":"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a"} Jul 13 21:29:01.544754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.544474 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w" event={"ID":"c99f16b1-d8f5-4b04-912b-0c35a9ea8054","Type":"ContainerDied","Data":"d69bd1a5977dbf5b6c6c0b6409b5e733d812b74aa832245db6fc9b5d7fe65de9"} Jul 13 21:29:01.544754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.544496 2564 scope.go:117] "RemoveContainer" containerID="37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a" Jul 13 21:29:01.577571 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.577538 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:29:01.579774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.579746 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4dw9w"] Jul 13 21:29:01.582418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.582068 2564 scope.go:117] "RemoveContainer" containerID="37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a" Jul 13 21:29:01.582510 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:29:01.582455 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a\": container with ID starting with 37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a not found: ID does not exist" containerID="37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a" Jul 13 21:29:01.582510 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.582486 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a"} err="failed to get container status \"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a\": rpc error: code = NotFound desc = could not find container \"37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a\": container with ID starting with 37b37fb9a78332cae196da64b17c9247be66c16d6a964549013c6724661b500a not found: ID does not exist" Jul 13 21:29:01.714741 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.714715 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:01.727267 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.727211 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:29:01.815681 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.815648 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:01.840653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840621 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.840838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840676 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.840838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840704 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.840838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840748 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.840838 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840785 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.841047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840884 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58tms\" (UniqueName: \"kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.841047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.840913 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert\") pod \"530eecca-2abb-4271-b41c-d00a266daa91\" (UID: \"530eecca-2abb-4271-b41c-d00a266daa91\") " Jul 13 21:29:01.841047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.841021 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca" (OuterVolumeSpecName: "service-ca") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:29:01.841259 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.841202 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config" (OuterVolumeSpecName: "console-config") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:29:01.841327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.841231 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.841327 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.841294 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:29:01.841428 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.841383 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:29:01.842956 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.842922 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:29:01.842956 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.842937 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms" (OuterVolumeSpecName: "kube-api-access-58tms") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "kube-api-access-58tms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:29:01.842956 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.842947 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "530eecca-2abb-4271-b41c-d00a266daa91" (UID: "530eecca-2abb-4271-b41c-d00a266daa91"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:29:01.942284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942247 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58tms\" (UniqueName: \"kubernetes.io/projected/530eecca-2abb-4271-b41c-d00a266daa91-kube-api-access-58tms\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.942284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942280 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-oauth-serving-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.942284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942289 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-console-config\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.942284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942297 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-oauth-config\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.942284 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942306 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/530eecca-2abb-4271-b41c-d00a266daa91-console-serving-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:01.942599 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:01.942315 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/530eecca-2abb-4271-b41c-d00a266daa91-trusted-ca-bundle\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:02.529840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.529808 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" path="/var/lib/kubelet/pods/c99f16b1-d8f5-4b04-912b-0c35a9ea8054/volumes" Jul 13 21:29:02.556919 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.556890 2564 generic.go:356] "Generic (PLEG): container finished" podID="530eecca-2abb-4271-b41c-d00a266daa91" containerID="4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247" exitCode=0 Jul 13 21:29:02.557296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.556944 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c6d8b9b-6wchx" Jul 13 21:29:02.557296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.556947 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c6d8b9b-6wchx" event={"ID":"530eecca-2abb-4271-b41c-d00a266daa91","Type":"ContainerDied","Data":"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247"} Jul 13 21:29:02.557296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.557096 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c6d8b9b-6wchx" event={"ID":"530eecca-2abb-4271-b41c-d00a266daa91","Type":"ContainerDied","Data":"49c30bcc9a69b25da0314260ecd975ad351f09e872bad23898337f3b99afa08d"} Jul 13 21:29:02.557296 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.557148 2564 scope.go:117] "RemoveContainer" containerID="4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247" Jul 13 21:29:02.565800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.565783 2564 scope.go:117] "RemoveContainer" containerID="4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247" Jul 13 21:29:02.566038 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:29:02.566022 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247\": container with ID starting with 4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247 not found: ID does not exist" containerID="4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247" Jul 13 21:29:02.566082 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.566050 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247"} err="failed to get container status \"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247\": rpc error: code = NotFound desc = could not find container \"4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247\": container with ID starting with 4ad7f17e927280f3f2d0423c5316906a1234c203bdd329e96a9f698421434247 not found: ID does not exist" Jul 13 21:29:02.573815 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.573794 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:29:02.577737 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:02.577710 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-696c6d8b9b-6wchx"] Jul 13 21:29:04.529495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:04.529464 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="530eecca-2abb-4271-b41c-d00a266daa91" path="/var/lib/kubelet/pods/530eecca-2abb-4271-b41c-d00a266daa91/volumes" Jul 13 21:29:06.040360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:06.040324 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:06.140905 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:06.140875 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:09.654604 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:09.654563 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:09.773053 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:09.773018 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:16.995837 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:16.995797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:17.199294 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:17.199262 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:18.718265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.718224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:18.728802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.728763 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:18.729208 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729193 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530eecca-2abb-4271-b41c-d00a266daa91" containerName="console" Jul 13 21:29:18.729208 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729209 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="530eecca-2abb-4271-b41c-d00a266daa91" containerName="console" Jul 13 21:29:18.729318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729220 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" containerName="manager" Jul 13 21:29:18.729318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729225 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" containerName="manager" Jul 13 21:29:18.729318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729279 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99f16b1-d8f5-4b04-912b-0c35a9ea8054" containerName="manager" Jul 13 21:29:18.729318 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.729288 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="530eecca-2abb-4271-b41c-d00a266daa91" containerName="console" Jul 13 21:29:18.757653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.757622 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:18.757809 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.757721 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.759933 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.759910 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 21:29:18.783948 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.783915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s98wc\" (UniqueName: \"kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.784112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.783974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.836302 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.836267 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:18.838798 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.838762 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:18.885029 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.884988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.885243 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.885073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s98wc\" (UniqueName: \"kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.885760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.885731 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.901929 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.901902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s98wc\" (UniqueName: \"kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc\") pod \"limitador-limitador-64c94fd64f-tj5pw\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:18.982558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:18.982472 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:19.068828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:19.068797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:19.257763 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:19.257680 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:19.261571 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:29:19.261533 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0608c76c_ff3c_4b88_8cf6_d20f80a74b06.slice/crio-beae754958329e967bcb74eb818dee6cb6b017a91e1ba177626f9df861b9c7fc WatchSource:0}: Error finding container beae754958329e967bcb74eb818dee6cb6b017a91e1ba177626f9df861b9c7fc: Status 404 returned error can't find the container with id beae754958329e967bcb74eb818dee6cb6b017a91e1ba177626f9df861b9c7fc Jul 13 21:29:19.653916 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:19.653882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" event={"ID":"0608c76c-ff3c-4b88-8cf6-d20f80a74b06","Type":"ContainerStarted","Data":"beae754958329e967bcb74eb818dee6cb6b017a91e1ba177626f9df861b9c7fc"} Jul 13 21:29:19.672958 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:19.672925 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:19.774994 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:19.774955 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:21.313408 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:21.313364 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:21.412321 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:21.412287 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:22.675576 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:22.675543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" event={"ID":"0608c76c-ff3c-4b88-8cf6-d20f80a74b06","Type":"ContainerStarted","Data":"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3"} Jul 13 21:29:22.676055 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:22.675631 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:22.692552 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:22.692505 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" podStartSLOduration=1.8321210049999999 podStartE2EDuration="4.692491129s" totalImagesPullingTime="2.860370124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:22.690230855 +0000 UTC m=+438.744765443" watchObservedRunningTime="2026-07-13 21:29:22.692491129 +0000 UTC m=+438.747025717" Jul 13 21:29:24.478467 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:24.478431 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:24.540589 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:24.540557 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:33.682262 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:33.682186 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:35.147322 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.147285 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:35.147729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.147475 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" podUID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" containerName="limitador" containerID="cri-o://3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3" gracePeriod=30 Jul 13 21:29:35.690381 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.690355 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:35.730211 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.730109 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file\") pod \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " Jul 13 21:29:35.730211 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.730193 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s98wc\" (UniqueName: \"kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc\") pod \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\" (UID: \"0608c76c-ff3c-4b88-8cf6-d20f80a74b06\") " Jul 13 21:29:35.730432 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.730401 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file" (OuterVolumeSpecName: "config-file") pod "0608c76c-ff3c-4b88-8cf6-d20f80a74b06" (UID: "0608c76c-ff3c-4b88-8cf6-d20f80a74b06"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:29:35.730554 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.730538 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-config-file\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:35.732185 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.732166 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc" (OuterVolumeSpecName: "kube-api-access-s98wc") pod "0608c76c-ff3c-4b88-8cf6-d20f80a74b06" (UID: "0608c76c-ff3c-4b88-8cf6-d20f80a74b06"). InnerVolumeSpecName "kube-api-access-s98wc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:29:35.746734 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.746706 2564 generic.go:356] "Generic (PLEG): container finished" podID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" containerID="3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3" exitCode=0 Jul 13 21:29:35.746876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.746753 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" Jul 13 21:29:35.746876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.746778 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" event={"ID":"0608c76c-ff3c-4b88-8cf6-d20f80a74b06","Type":"ContainerDied","Data":"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3"} Jul 13 21:29:35.746876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.746843 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-tj5pw" event={"ID":"0608c76c-ff3c-4b88-8cf6-d20f80a74b06","Type":"ContainerDied","Data":"beae754958329e967bcb74eb818dee6cb6b017a91e1ba177626f9df861b9c7fc"} Jul 13 21:29:35.746876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.746864 2564 scope.go:117] "RemoveContainer" containerID="3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3" Jul 13 21:29:35.755592 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.755575 2564 scope.go:117] "RemoveContainer" containerID="3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3" Jul 13 21:29:35.755832 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:29:35.755814 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3\": container with ID starting with 3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3 not found: ID does not exist" containerID="3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3" Jul 13 21:29:35.755878 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.755841 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3"} err="failed to get container status \"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3\": rpc error: code = NotFound desc = could not find container \"3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3\": container with ID starting with 3ee9055f3c7312aaab5efde5859da9de4d0903ab668c90f0ce907f4261ad43a3 not found: ID does not exist" Jul 13 21:29:35.767541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.767519 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:35.769552 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.769534 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-tj5pw"] Jul 13 21:29:35.830984 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:35.830955 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s98wc\" (UniqueName: \"kubernetes.io/projected/0608c76c-ff3c-4b88-8cf6-d20f80a74b06-kube-api-access-s98wc\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:29:36.528670 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:36.528629 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" path="/var/lib/kubelet/pods/0608c76c-ff3c-4b88-8cf6-d20f80a74b06/volumes" Jul 13 21:29:39.967289 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:39.967248 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:40.068900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.068872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:40.360628 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.360591 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5"] Jul 13 21:29:40.361031 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.361012 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" containerName="limitador" Jul 13 21:29:40.361142 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.361034 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" containerName="limitador" Jul 13 21:29:40.361212 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.361149 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0608c76c-ff3c-4b88-8cf6-d20f80a74b06" containerName="limitador" Jul 13 21:29:40.365710 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.365687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.368069 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.368046 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 21:29:40.368218 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.368047 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 21:29:40.368767 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.368745 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4ngsl\"" Jul 13 21:29:40.368856 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.368747 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 21:29:40.373081 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.373059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5"] Jul 13 21:29:40.398754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.398720 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:40.470846 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.470809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pftk9\" (UniqueName: \"kubernetes.io/projected/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-kube-api-access-pftk9\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.471022 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.470895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-data\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.572075 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.572039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pftk9\" (UniqueName: \"kubernetes.io/projected/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-kube-api-access-pftk9\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.572419 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.572113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-data\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.572528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.572505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-data\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.588037 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.588007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pftk9\" (UniqueName: \"kubernetes.io/projected/fd26bcf8-936c-4027-a4d6-8df6c55f33c6-kube-api-access-pftk9\") pod \"postgres-69fb4594cf-rkhg5\" (UID: \"fd26bcf8-936c-4027-a4d6-8df6c55f33c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.677710 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.677615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:40.809746 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:40.809722 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5"] Jul 13 21:29:40.812310 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:29:40.812283 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd26bcf8_936c_4027_a4d6_8df6c55f33c6.slice/crio-763b792a4eec6964930a12ea231a96fb491ad57c0ed9b831f0bee01881d0f0c2 WatchSource:0}: Error finding container 763b792a4eec6964930a12ea231a96fb491ad57c0ed9b831f0bee01881d0f0c2: Status 404 returned error can't find the container with id 763b792a4eec6964930a12ea231a96fb491ad57c0ed9b831f0bee01881d0f0c2 Jul 13 21:29:41.784452 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:41.784401 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" event={"ID":"fd26bcf8-936c-4027-a4d6-8df6c55f33c6","Type":"ContainerStarted","Data":"763b792a4eec6964930a12ea231a96fb491ad57c0ed9b831f0bee01881d0f0c2"} Jul 13 21:29:45.810718 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:45.810680 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" event={"ID":"fd26bcf8-936c-4027-a4d6-8df6c55f33c6","Type":"ContainerStarted","Data":"447110b9b70204672225863acd4b6dc5090b26c782b00343ddb1bf76121f50b3"} Jul 13 21:29:45.811111 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:45.810818 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:45.829454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:45.829389 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" podStartSLOduration=1.212912334 podStartE2EDuration="5.829365159s" totalImagesPullingTime="4.616452825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:45.825161297 +0000 UTC m=+461.879695875" watchObservedRunningTime="2026-07-13 21:29:45.829365159 +0000 UTC m=+461.883899745" Jul 13 21:29:51.845508 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:51.845479 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rkhg5" Jul 13 21:29:52.150311 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:52.150224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:52.329422 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:52.329394 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:58.873150 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.873071 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:58.885405 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.885374 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:29:58.888881 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.888859 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:58.893743 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.893710 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wsmxx\"" Jul 13 21:29:58.893906 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.893890 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 21:29:58.894447 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.894433 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 21:29:58.903563 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.903537 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:29:58.975400 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:58.975367 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:59.046523 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.046486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7xfw\" (UniqueName: \"kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.046696 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.046534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.066629 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.066595 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:29:59.067201 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.067171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:59.070200 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.070181 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.079938 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.079912 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:29:59.138331 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.138247 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:29:59.147416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.147393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7xfw\" (UniqueName: \"kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.147528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.147426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.149929 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.149902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.155953 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.155930 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7xfw\" (UniqueName: \"kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw\") pod \"maas-controller-7cccfc6f5d-kdwxl\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.198441 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.198407 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:29:59.248849 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.248814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjvb\" (UniqueName: \"kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.248849 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.248986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.334065 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.334038 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:29:59.335464 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:29:59.335438 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b83cc7_4456_4c88_993e_1eb7a068a1af.slice/crio-bf6fbfc2088e1fa4c91dcc395a8eb72d83b16cc1bce777702ced9d65928f0180 WatchSource:0}: Error finding container bf6fbfc2088e1fa4c91dcc395a8eb72d83b16cc1bce777702ced9d65928f0180: Status 404 returned error can't find the container with id bf6fbfc2088e1fa4c91dcc395a8eb72d83b16cc1bce777702ced9d65928f0180 Jul 13 21:29:59.350354 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.350332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.350481 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.350378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjvb\" (UniqueName: \"kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.352605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.352587 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.358696 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.358677 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjvb\" (UniqueName: \"kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb\") pod \"maas-controller-77f465dbd8-ccwbr\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.381541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.381516 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:29:59.519307 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.519283 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:29:59.521241 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:29:59.521214 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2927db80_f896_43c9_bb91_586e595a0959.slice/crio-430b099068a0fc430f0c9bac55f023daa482a0a0aa4fd280640692cd721d382c WatchSource:0}: Error finding container 430b099068a0fc430f0c9bac55f023daa482a0a0aa4fd280640692cd721d382c: Status 404 returned error can't find the container with id 430b099068a0fc430f0c9bac55f023daa482a0a0aa4fd280640692cd721d382c Jul 13 21:29:59.889555 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.889520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" event={"ID":"77b83cc7-4456-4c88-993e-1eb7a068a1af","Type":"ContainerStarted","Data":"bf6fbfc2088e1fa4c91dcc395a8eb72d83b16cc1bce777702ced9d65928f0180"} Jul 13 21:29:59.891034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:29:59.891001 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" event={"ID":"2927db80-f896-43c9-bb91-586e595a0959","Type":"ContainerStarted","Data":"430b099068a0fc430f0c9bac55f023daa482a0a0aa4fd280640692cd721d382c"} Jul 13 21:30:01.792868 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.792776 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:01.890330 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.890288 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:01.908436 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.908358 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" event={"ID":"2927db80-f896-43c9-bb91-586e595a0959","Type":"ContainerStarted","Data":"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874"} Jul 13 21:30:01.908645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.908527 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:30:01.910041 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.910018 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" event={"ID":"77b83cc7-4456-4c88-993e-1eb7a068a1af","Type":"ContainerStarted","Data":"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d"} Jul 13 21:30:01.910203 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.910097 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:30:01.929614 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.929557 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" podStartSLOduration=0.896837695 podStartE2EDuration="2.929540641s" totalImagesPullingTime="2.032702946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:01.926105105 +0000 UTC m=+477.980639694" watchObservedRunningTime="2026-07-13 21:30:01.929540641 +0000 UTC m=+477.984075242" Jul 13 21:30:01.946240 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:01.946172 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" podStartSLOduration=1.738858019 podStartE2EDuration="3.946153989s" totalImagesPullingTime="2.20729597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:01.944099555 +0000 UTC m=+477.998634145" watchObservedRunningTime="2026-07-13 21:30:01.946153989 +0000 UTC m=+478.000688576" Jul 13 21:30:12.918470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:12.918440 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:30:12.919039 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:12.918520 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:30:12.981431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:12.981325 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:30:12.982241 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:12.981613 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" podUID="77b83cc7-4456-4c88-993e-1eb7a068a1af" containerName="manager" containerID="cri-o://0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d" gracePeriod=30 Jul 13 21:30:13.234046 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.234020 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:30:13.276641 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.276612 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert\") pod \"77b83cc7-4456-4c88-993e-1eb7a068a1af\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " Jul 13 21:30:13.276800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.276664 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7xfw\" (UniqueName: \"kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw\") pod \"77b83cc7-4456-4c88-993e-1eb7a068a1af\" (UID: \"77b83cc7-4456-4c88-993e-1eb7a068a1af\") " Jul 13 21:30:13.278863 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.278831 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw" (OuterVolumeSpecName: "kube-api-access-d7xfw") pod "77b83cc7-4456-4c88-993e-1eb7a068a1af" (UID: "77b83cc7-4456-4c88-993e-1eb7a068a1af"). InnerVolumeSpecName "kube-api-access-d7xfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:30:13.278863 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.278853 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "77b83cc7-4456-4c88-993e-1eb7a068a1af" (UID: "77b83cc7-4456-4c88-993e-1eb7a068a1af"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:30:13.377457 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.377423 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77b83cc7-4456-4c88-993e-1eb7a068a1af-webhook-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:30:13.377457 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.377453 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7xfw\" (UniqueName: \"kubernetes.io/projected/77b83cc7-4456-4c88-993e-1eb7a068a1af-kube-api-access-d7xfw\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:30:13.978188 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.978148 2564 generic.go:356] "Generic (PLEG): container finished" podID="77b83cc7-4456-4c88-993e-1eb7a068a1af" containerID="0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d" exitCode=0 Jul 13 21:30:13.978596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.978241 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" Jul 13 21:30:13.978596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.978261 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" event={"ID":"77b83cc7-4456-4c88-993e-1eb7a068a1af","Type":"ContainerDied","Data":"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d"} Jul 13 21:30:13.978596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.978305 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cccfc6f5d-kdwxl" event={"ID":"77b83cc7-4456-4c88-993e-1eb7a068a1af","Type":"ContainerDied","Data":"bf6fbfc2088e1fa4c91dcc395a8eb72d83b16cc1bce777702ced9d65928f0180"} Jul 13 21:30:13.978596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.978325 2564 scope.go:117] "RemoveContainer" containerID="0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d" Jul 13 21:30:13.987828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.987617 2564 scope.go:117] "RemoveContainer" containerID="0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d" Jul 13 21:30:13.987902 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:30:13.987884 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d\": container with ID starting with 0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d not found: ID does not exist" containerID="0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d" Jul 13 21:30:13.987938 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.987908 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d"} err="failed to get container status \"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d\": rpc error: code = NotFound desc = could not find container \"0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d\": container with ID starting with 0fdf9b5640d17cb70e918e1b4dd1d6b3b5e837d0ffdaa6646ed302332960088d not found: ID does not exist" Jul 13 21:30:13.999930 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:13.999908 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:30:14.003737 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:14.003718 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cccfc6f5d-kdwxl"] Jul 13 21:30:14.528635 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:14.528603 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b83cc7-4456-4c88-993e-1eb7a068a1af" path="/var/lib/kubelet/pods/77b83cc7-4456-4c88-993e-1eb7a068a1af/volumes" Jul 13 21:30:33.316216 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:33.316171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:33.419009 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:33.418957 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:33.882360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:33.882320 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:34.025635 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.025598 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj"] Jul 13 21:30:34.026003 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.025990 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b83cc7-4456-4c88-993e-1eb7a068a1af" containerName="manager" Jul 13 21:30:34.026003 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.026004 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b83cc7-4456-4c88-993e-1eb7a068a1af" containerName="manager" Jul 13 21:30:34.026091 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.026073 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b83cc7-4456-4c88-993e-1eb7a068a1af" containerName="manager" Jul 13 21:30:34.029268 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.029252 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.031638 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.031609 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 21:30:34.031783 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.031748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nq9m7\"" Jul 13 21:30:34.042859 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.042831 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj"] Jul 13 21:30:34.174884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.174783 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-maas-api-tls\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.174884 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.174840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-openshift-service-ca\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.175100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.174897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb5fl\" (UniqueName: \"kubernetes.io/projected/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-kube-api-access-hb5fl\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.199857 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.199823 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:34.275416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.275383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-maas-api-tls\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.275599 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.275431 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-openshift-service-ca\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.275599 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.275540 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb5fl\" (UniqueName: \"kubernetes.io/projected/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-kube-api-access-hb5fl\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.276072 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.276051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-openshift-service-ca\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.277981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.277954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-maas-api-tls\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.292045 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.292022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb5fl\" (UniqueName: \"kubernetes.io/projected/8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8-kube-api-access-hb5fl\") pod \"maas-api-5df54695bf-kkqbj\" (UID: \"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8\") " pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.339920 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.339881 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:34.472577 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.472542 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:34.501910 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.501885 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj"] Jul 13 21:30:34.504276 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:30:34.504246 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2231d1_4fbf_4ee0_a91b_8e63c493a8d8.slice/crio-fe82d311acdc758d924b171e77e64a51165b92986fc363e155f833c62dfbc778 WatchSource:0}: Error finding container fe82d311acdc758d924b171e77e64a51165b92986fc363e155f833c62dfbc778: Status 404 returned error can't find the container with id fe82d311acdc758d924b171e77e64a51165b92986fc363e155f833c62dfbc778 Jul 13 21:30:34.570063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.570024 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:34.574205 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.574180 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-cff55"] Jul 13 21:30:34.578802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.578783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.582361 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.582343 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 21:30:34.582481 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.582462 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g7zmg\"" Jul 13 21:30:34.591191 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.591166 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-cff55"] Jul 13 21:30:34.678191 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.678161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eef1f64f-57af-4854-a4a6-c2405a59d24a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.678352 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.678240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px27b\" (UniqueName: \"kubernetes.io/projected/eef1f64f-57af-4854-a4a6-c2405a59d24a-kube-api-access-px27b\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.779032 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.778995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px27b\" (UniqueName: \"kubernetes.io/projected/eef1f64f-57af-4854-a4a6-c2405a59d24a-kube-api-access-px27b\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.779251 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.779100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eef1f64f-57af-4854-a4a6-c2405a59d24a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.779824 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.779799 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eef1f64f-57af-4854-a4a6-c2405a59d24a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.790210 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.790186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px27b\" (UniqueName: \"kubernetes.io/projected/eef1f64f-57af-4854-a4a6-c2405a59d24a-kube-api-access-px27b\") pod \"payload-pre-processing-74584bb9f8-cff55\" (UID: \"eef1f64f-57af-4854-a4a6-c2405a59d24a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:34.891379 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:34.891328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:35.029751 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:35.029696 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-cff55"] Jul 13 21:30:35.030833 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:30:35.030812 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef1f64f_57af_4854_a4a6_c2405a59d24a.slice/crio-0e5b0d36f03a1f8d10146647483ae2a009c2f1435cea4d8ee16d44cd4689cde6 WatchSource:0}: Error finding container 0e5b0d36f03a1f8d10146647483ae2a009c2f1435cea4d8ee16d44cd4689cde6: Status 404 returned error can't find the container with id 0e5b0d36f03a1f8d10146647483ae2a009c2f1435cea4d8ee16d44cd4689cde6 Jul 13 21:30:35.093278 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:35.093238 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" event={"ID":"eef1f64f-57af-4854-a4a6-c2405a59d24a","Type":"ContainerStarted","Data":"0e5b0d36f03a1f8d10146647483ae2a009c2f1435cea4d8ee16d44cd4689cde6"} Jul 13 21:30:35.094549 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:35.094518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" event={"ID":"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8","Type":"ContainerStarted","Data":"fe82d311acdc758d924b171e77e64a51165b92986fc363e155f833c62dfbc778"} Jul 13 21:30:37.107045 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.107006 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" event={"ID":"eef1f64f-57af-4854-a4a6-c2405a59d24a","Type":"ContainerStarted","Data":"c4f24ce5ff8a94c8c73fe483ba17ef4a92a3ca30f286d0fd7931f6080fd8557a"} Jul 13 21:30:37.107522 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.107114 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:37.108597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.108571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" event={"ID":"8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8","Type":"ContainerStarted","Data":"5f25a95c11342eb556df4b799906e7d79358414cd885b26e12f2e9a85c724955"} Jul 13 21:30:37.108718 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.108703 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:37.123800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.123755 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" podStartSLOduration=1.702414796 podStartE2EDuration="3.123739373s" totalImagesPullingTime="1.421324577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:37.121705749 +0000 UTC m=+513.176240337" watchObservedRunningTime="2026-07-13 21:30:37.123739373 +0000 UTC m=+513.178273959" Jul 13 21:30:37.138773 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:37.138724 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" podStartSLOduration=1.193011629 podStartE2EDuration="3.138707746s" totalImagesPullingTime="1.945696117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:37.135864261 +0000 UTC m=+513.190398888" watchObservedRunningTime="2026-07-13 21:30:37.138707746 +0000 UTC m=+513.193242334" Jul 13 21:30:42.183081 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:42.183047 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:43.119383 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:43.119356 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df54695bf-kkqbj" Jul 13 21:30:44.626193 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:44.626148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:44.720353 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:44.720317 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:48.115813 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:48.115789 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-cff55" Jul 13 21:30:53.695557 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.695519 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc"] Jul 13 21:30:53.700435 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.700417 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.703648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.703628 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rkxvv\"" Jul 13 21:30:53.703648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.703640 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 21:30:53.703802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.703642 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 21:30:53.703802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.703685 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 21:30:53.709914 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.709892 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc"] Jul 13 21:30:53.718860 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.718821 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:30:53.752812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.752983 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.752983 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.752983 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbs4l\" (UniqueName: \"kubernetes.io/projected/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-kube-api-access-dbs4l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.752983 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.753159 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.752991 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854516 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854693 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbs4l\" (UniqueName: \"kubernetes.io/projected/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-kube-api-access-dbs4l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854693 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854813 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854868 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.854918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.854885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.855135 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.855089 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.855298 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.855276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.855448 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.855419 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.857431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.857410 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.857996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.857962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:53.861683 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:53.861657 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbs4l\" (UniqueName: \"kubernetes.io/projected/6a23f191-6c8f-4b18-b6bb-bce9e038dc42-kube-api-access-dbs4l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc\" (UID: \"6a23f191-6c8f-4b18-b6bb-bce9e038dc42\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:54.011392 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:54.011293 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:30:54.159355 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:54.159332 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc"] Jul 13 21:30:54.161338 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:30:54.161314 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a23f191_6c8f_4b18_b6bb_bce9e038dc42.slice/crio-06404c145c7ab2d631463bbccc80e7d16c1ff55aa33b07b117ecf837dc54d516 WatchSource:0}: Error finding container 06404c145c7ab2d631463bbccc80e7d16c1ff55aa33b07b117ecf837dc54d516: Status 404 returned error can't find the container with id 06404c145c7ab2d631463bbccc80e7d16c1ff55aa33b07b117ecf837dc54d516 Jul 13 21:30:54.202307 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:54.202270 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" event={"ID":"6a23f191-6c8f-4b18-b6bb-bce9e038dc42","Type":"ContainerStarted","Data":"06404c145c7ab2d631463bbccc80e7d16c1ff55aa33b07b117ecf837dc54d516"} Jul 13 21:30:56.216778 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:30:56.216679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" event={"ID":"6a23f191-6c8f-4b18-b6bb-bce9e038dc42","Type":"ContainerStarted","Data":"48e378aef9277f7ea85d71f3143e882fcb0205c4eeeadaa3f85c32bd3c4e4a50"} Jul 13 21:31:04.012291 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.012253 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:31:04.012786 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.012356 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:31:04.024726 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.024701 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:31:04.042345 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.042294 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" podStartSLOduration=9.895410955 podStartE2EDuration="11.042276924s" totalImagesPullingTime="1.146865969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:56.237431523 +0000 UTC m=+532.291966128" watchObservedRunningTime="2026-07-13 21:31:04.042276924 +0000 UTC m=+540.096811573" Jul 13 21:31:04.269732 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.269707 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc" Jul 13 21:31:04.550370 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.550284 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:04.651211 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:04.651177 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:10.922557 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:10.922523 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:15.887656 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:15.887621 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn"] Jul 13 21:31:15.893457 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:15.893436 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:15.896703 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:15.896676 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 21:31:15.899557 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:15.899533 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn"] Jul 13 21:31:15.983550 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:15.983516 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:16.066201 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066159 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.066384 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59694b11-e8c3-48cf-880a-025d2bee8836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.066384 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt27j\" (UniqueName: \"kubernetes.io/projected/59694b11-e8c3-48cf-880a-025d2bee8836-kube-api-access-qt27j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.066473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.066473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066465 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.066543 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.066504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167420 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167420 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167626 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167626 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59694b11-e8c3-48cf-880a-025d2bee8836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167626 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt27j\" (UniqueName: \"kubernetes.io/projected/59694b11-e8c3-48cf-880a-025d2bee8836-kube-api-access-qt27j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167626 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167833 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167808 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167881 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167832 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.167940 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.167877 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.169808 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.169780 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59694b11-e8c3-48cf-880a-025d2bee8836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.169967 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.169948 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59694b11-e8c3-48cf-880a-025d2bee8836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.175356 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.175338 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt27j\" (UniqueName: \"kubernetes.io/projected/59694b11-e8c3-48cf-880a-025d2bee8836-kube-api-access-qt27j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wtpfn\" (UID: \"59694b11-e8c3-48cf-880a-025d2bee8836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.204361 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.204331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:16.344864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:16.344838 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn"] Jul 13 21:31:16.346790 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:31:16.346765 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59694b11_e8c3_48cf_880a_025d2bee8836.slice/crio-dbb06170cb4f383e541cdc9a3cfed46b63834a87c4419be6b9ea43fc2299fb5f WatchSource:0}: Error finding container dbb06170cb4f383e541cdc9a3cfed46b63834a87c4419be6b9ea43fc2299fb5f: Status 404 returned error can't find the container with id dbb06170cb4f383e541cdc9a3cfed46b63834a87c4419be6b9ea43fc2299fb5f Jul 13 21:31:17.334818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:17.334781 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" event={"ID":"59694b11-e8c3-48cf-880a-025d2bee8836","Type":"ContainerStarted","Data":"f58f8a946b2aae8b128ba356bfe5b80da1fda7b3495c01e554063719f7b1544a"} Jul 13 21:31:17.335223 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:17.334840 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" event={"ID":"59694b11-e8c3-48cf-880a-025d2bee8836","Type":"ContainerStarted","Data":"dbb06170cb4f383e541cdc9a3cfed46b63834a87c4419be6b9ea43fc2299fb5f"} Jul 13 21:31:17.354047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:17.353998 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" podStartSLOduration=2.175453867 podStartE2EDuration="2.353981633s" totalImagesPullingTime="178.527766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:31:17.351353425 +0000 UTC m=+553.405888013" watchObservedRunningTime="2026-07-13 21:31:17.353981633 +0000 UTC m=+553.408516219" Jul 13 21:31:26.204645 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:26.204602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:26.205197 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:26.204677 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:26.217189 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:26.217165 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:26.395719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:26.395689 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wtpfn" Jul 13 21:31:28.689933 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.689903 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55"] Jul 13 21:31:28.695080 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.695052 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55"] Jul 13 21:31:28.695938 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.695276 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.699581 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.698583 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 21:31:28.719289 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.719261 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:28.778762 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9rg2\" (UniqueName: \"kubernetes.io/projected/845b5989-2469-4248-959b-1de00f8b52e6-kube-api-access-f9rg2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.778762 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.778966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845b5989-2469-4248-959b-1de00f8b52e6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.778966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778846 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.778966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778872 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.778966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.778901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880102 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9rg2\" (UniqueName: \"kubernetes.io/projected/845b5989-2469-4248-959b-1de00f8b52e6-kube-api-access-f9rg2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880273 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845b5989-2469-4248-959b-1de00f8b52e6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880558 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880539 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880637 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880571 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.880637 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.880605 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.882504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.882484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845b5989-2469-4248-959b-1de00f8b52e6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.882689 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.882674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845b5989-2469-4248-959b-1de00f8b52e6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:28.890515 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:28.890492 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9rg2\" (UniqueName: \"kubernetes.io/projected/845b5989-2469-4248-959b-1de00f8b52e6-kube-api-access-f9rg2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55\" (UID: \"845b5989-2469-4248-959b-1de00f8b52e6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:29.010593 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:29.010495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:29.175774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:29.175671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55"] Jul 13 21:31:29.178406 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:31:29.178371 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845b5989_2469_4248_959b_1de00f8b52e6.slice/crio-53b774bd204d58bf3948b0978791d927156541a515951b0db5b2b55c1b28c145 WatchSource:0}: Error finding container 53b774bd204d58bf3948b0978791d927156541a515951b0db5b2b55c1b28c145: Status 404 returned error can't find the container with id 53b774bd204d58bf3948b0978791d927156541a515951b0db5b2b55c1b28c145 Jul 13 21:31:29.401534 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:29.401508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" event={"ID":"845b5989-2469-4248-959b-1de00f8b52e6","Type":"ContainerStarted","Data":"53b774bd204d58bf3948b0978791d927156541a515951b0db5b2b55c1b28c145"} Jul 13 21:31:30.408497 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:30.408463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" event={"ID":"845b5989-2469-4248-959b-1de00f8b52e6","Type":"ContainerStarted","Data":"4ae298c7234b44d267b5fd302b67701ce1ee9dadbd8a8a3d9635a1fd319c74e8"} Jul 13 21:31:30.426449 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:30.426381 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" podStartSLOduration=2.251184067 podStartE2EDuration="2.426364412s" totalImagesPullingTime="175.180345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:31:30.424269463 +0000 UTC m=+566.478804071" watchObservedRunningTime="2026-07-13 21:31:30.426364412 +0000 UTC m=+566.480898998" Jul 13 21:31:33.366541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:33.366504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:37.615861 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:37.615826 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:31:39.010945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:39.010907 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:39.011381 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:39.010974 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:39.023895 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:39.023868 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:31:39.471343 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:31:39.471314 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55" Jul 13 21:32:28.628961 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:32:28.628869 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:32:29.828981 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:32:29.828946 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:32:38.053706 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:32:38.053676 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:32:41.087971 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:32:41.087935 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:33:39.425536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.425504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:33:39.540937 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.540905 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:33:39.541177 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.541127 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" podUID="2927db80-f896-43c9-bb91-586e595a0959" containerName="manager" containerID="cri-o://866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874" gracePeriod=30 Jul 13 21:33:39.792282 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.792254 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:33:39.911782 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.911744 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert\") pod \"2927db80-f896-43c9-bb91-586e595a0959\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " Jul 13 21:33:39.911957 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.911799 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjjvb\" (UniqueName: \"kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb\") pod \"2927db80-f896-43c9-bb91-586e595a0959\" (UID: \"2927db80-f896-43c9-bb91-586e595a0959\") " Jul 13 21:33:39.913966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.913930 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb" (OuterVolumeSpecName: "kube-api-access-rjjvb") pod "2927db80-f896-43c9-bb91-586e595a0959" (UID: "2927db80-f896-43c9-bb91-586e595a0959"). InnerVolumeSpecName "kube-api-access-rjjvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:33:39.913966 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:39.913950 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2927db80-f896-43c9-bb91-586e595a0959" (UID: "2927db80-f896-43c9-bb91-586e595a0959"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:33:40.013546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.013517 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2927db80-f896-43c9-bb91-586e595a0959-webhook-cert\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:33:40.013546 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.013548 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjjvb\" (UniqueName: \"kubernetes.io/projected/2927db80-f896-43c9-bb91-586e595a0959-kube-api-access-rjjvb\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:33:40.134667 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.134631 2564 generic.go:356] "Generic (PLEG): container finished" podID="2927db80-f896-43c9-bb91-586e595a0959" containerID="866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874" exitCode=0 Jul 13 21:33:40.134840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.134675 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" Jul 13 21:33:40.134840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.134715 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" event={"ID":"2927db80-f896-43c9-bb91-586e595a0959","Type":"ContainerDied","Data":"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874"} Jul 13 21:33:40.134840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.134793 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ccwbr" event={"ID":"2927db80-f896-43c9-bb91-586e595a0959","Type":"ContainerDied","Data":"430b099068a0fc430f0c9bac55f023daa482a0a0aa4fd280640692cd721d382c"} Jul 13 21:33:40.134840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.134816 2564 scope.go:117] "RemoveContainer" containerID="866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874" Jul 13 21:33:40.144326 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.144307 2564 scope.go:117] "RemoveContainer" containerID="866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874" Jul 13 21:33:40.144586 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:33:40.144567 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874\": container with ID starting with 866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874 not found: ID does not exist" containerID="866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874" Jul 13 21:33:40.144637 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.144596 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874"} err="failed to get container status \"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874\": rpc error: code = NotFound desc = could not find container \"866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874\": container with ID starting with 866db115b63b90311fec34371e3093e4b229948f34bd518c120a610c0d5f2874 not found: ID does not exist" Jul 13 21:33:40.155885 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.155859 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:33:40.162047 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.162026 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ccwbr"] Jul 13 21:33:40.531004 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.530971 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2927db80-f896-43c9-bb91-586e595a0959" path="/var/lib/kubelet/pods/2927db80-f896-43c9-bb91-586e595a0959/volumes" Jul 13 21:33:40.825773 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.825691 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:33:40.933274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.933240 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ljrrr"] Jul 13 21:33:40.933675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.933659 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2927db80-f896-43c9-bb91-586e595a0959" containerName="manager" Jul 13 21:33:40.933675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.933675 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2927db80-f896-43c9-bb91-586e595a0959" containerName="manager" Jul 13 21:33:40.933843 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.933753 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2927db80-f896-43c9-bb91-586e595a0959" containerName="manager" Jul 13 21:33:40.938340 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.938322 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:40.941400 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.941377 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 21:33:40.941400 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.941395 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 21:33:40.941548 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.941447 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wsmxx\"" Jul 13 21:33:40.946721 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:40.946698 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ljrrr"] Jul 13 21:33:41.022850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.022821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d0e2e-b098-4c98-9d7e-dec12b6439fd-webhook-cert\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.023035 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.022892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dlvp\" (UniqueName: \"kubernetes.io/projected/632d0e2e-b098-4c98-9d7e-dec12b6439fd-kube-api-access-9dlvp\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.123528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.123420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dlvp\" (UniqueName: \"kubernetes.io/projected/632d0e2e-b098-4c98-9d7e-dec12b6439fd-kube-api-access-9dlvp\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.123528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.123504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d0e2e-b098-4c98-9d7e-dec12b6439fd-webhook-cert\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.126034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.126008 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d0e2e-b098-4c98-9d7e-dec12b6439fd-webhook-cert\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.131470 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.131439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dlvp\" (UniqueName: \"kubernetes.io/projected/632d0e2e-b098-4c98-9d7e-dec12b6439fd-kube-api-access-9dlvp\") pod \"maas-controller-77f465dbd8-ljrrr\" (UID: \"632d0e2e-b098-4c98-9d7e-dec12b6439fd\") " pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.249705 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.249643 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:41.395013 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.394978 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77f465dbd8-ljrrr"] Jul 13 21:33:41.396508 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:33:41.396482 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632d0e2e_b098_4c98_9d7e_dec12b6439fd.slice/crio-13f5a92c5a1d63caed6a3f7e72bbb1aa56c5c278f3e7b67f6b4a5d43a73f59bb WatchSource:0}: Error finding container 13f5a92c5a1d63caed6a3f7e72bbb1aa56c5c278f3e7b67f6b4a5d43a73f59bb: Status 404 returned error can't find the container with id 13f5a92c5a1d63caed6a3f7e72bbb1aa56c5c278f3e7b67f6b4a5d43a73f59bb Jul 13 21:33:41.398005 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:41.397985 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:33:42.150395 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:42.150293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" event={"ID":"632d0e2e-b098-4c98-9d7e-dec12b6439fd","Type":"ContainerStarted","Data":"541d08295240647a6e703dd9034498805f4a2fde2eced727a909acec3b7820ca"} Jul 13 21:33:42.150395 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:42.150361 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" event={"ID":"632d0e2e-b098-4c98-9d7e-dec12b6439fd","Type":"ContainerStarted","Data":"13f5a92c5a1d63caed6a3f7e72bbb1aa56c5c278f3e7b67f6b4a5d43a73f59bb"} Jul 13 21:33:42.150831 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:42.150508 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:42.168745 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:42.168695 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" podStartSLOduration=1.752111341 podStartE2EDuration="2.168679607s" totalImagesPullingTime="416.568266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:42.166274657 +0000 UTC m=+698.220809246" watchObservedRunningTime="2026-07-13 21:33:42.168679607 +0000 UTC m=+698.223214194" Jul 13 21:33:53.159382 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:53.159352 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77f465dbd8-ljrrr" Jul 13 21:33:53.358233 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:33:53.358193 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:14.144689 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:14.144639 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:14.247320 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:14.247284 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:14.391476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:14.391440 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:19.968202 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:19.968163 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:20.119254 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:20.119208 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:20.214862 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:20.214831 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:20.356880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:20.356847 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:26.039719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:26.039682 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:34:26.139877 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:34:26.139844 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:35:05.543758 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:35:05.543719 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:35:13.394756 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:35:13.394722 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:36:50.569025 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:36:50.568992 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:38:22.978036 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:22.978001 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:38:22.980577 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:22.978208 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" podUID="3563a519-5ca9-48a8-b935-bd55e07ac803" containerName="manager" containerID="cri-o://ad1c6cd46342781a2fd4489f32d629c05d0d2340b07184c2609d2a249fdc5e56" gracePeriod=10 Jul 13 21:38:25.783086 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:25.783051 2564 generic.go:356] "Generic (PLEG): container finished" podID="3563a519-5ca9-48a8-b935-bd55e07ac803" containerID="ad1c6cd46342781a2fd4489f32d629c05d0d2340b07184c2609d2a249fdc5e56" exitCode=0 Jul 13 21:38:25.783472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:25.783140 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" event={"ID":"3563a519-5ca9-48a8-b935-bd55e07ac803","Type":"ContainerDied","Data":"ad1c6cd46342781a2fd4489f32d629c05d0d2340b07184c2609d2a249fdc5e56"} Jul 13 21:38:25.934498 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:25.934468 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:38:26.044472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.044388 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume\") pod \"3563a519-5ca9-48a8-b935-bd55e07ac803\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " Jul 13 21:38:26.044614 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.044498 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5lzj\" (UniqueName: \"kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj\") pod \"3563a519-5ca9-48a8-b935-bd55e07ac803\" (UID: \"3563a519-5ca9-48a8-b935-bd55e07ac803\") " Jul 13 21:38:26.044859 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.044829 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3563a519-5ca9-48a8-b935-bd55e07ac803" (UID: "3563a519-5ca9-48a8-b935-bd55e07ac803"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 21:38:26.046477 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.046459 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj" (OuterVolumeSpecName: "kube-api-access-s5lzj") pod "3563a519-5ca9-48a8-b935-bd55e07ac803" (UID: "3563a519-5ca9-48a8-b935-bd55e07ac803"). InnerVolumeSpecName "kube-api-access-s5lzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:38:26.145279 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.145243 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5lzj\" (UniqueName: \"kubernetes.io/projected/3563a519-5ca9-48a8-b935-bd55e07ac803-kube-api-access-s5lzj\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:38:26.145279 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.145270 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3563a519-5ca9-48a8-b935-bd55e07ac803-extensions-socket-volume\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:38:26.790055 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.789965 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" Jul 13 21:38:26.790055 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.790002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7" event={"ID":"3563a519-5ca9-48a8-b935-bd55e07ac803","Type":"ContainerDied","Data":"9a645dc8854cf4f64947eaf8fd051739d2e7fe28a2f6bf24a48b2e512ea98f41"} Jul 13 21:38:26.790590 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.790074 2564 scope.go:117] "RemoveContainer" containerID="ad1c6cd46342781a2fd4489f32d629c05d0d2340b07184c2609d2a249fdc5e56" Jul 13 21:38:26.807962 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.807901 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:38:26.810928 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:26.810906 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gb7q7"] Jul 13 21:38:28.529435 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:38:28.529393 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3563a519-5ca9-48a8-b935-bd55e07ac803" path="/var/lib/kubelet/pods/3563a519-5ca9-48a8-b935-bd55e07ac803/volumes" Jul 13 21:39:24.087346 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.087305 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs"] Jul 13 21:39:24.087935 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.087912 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3563a519-5ca9-48a8-b935-bd55e07ac803" containerName="manager" Jul 13 21:39:24.088018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.087939 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3563a519-5ca9-48a8-b935-bd55e07ac803" containerName="manager" Jul 13 21:39:24.088085 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.088071 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3563a519-5ca9-48a8-b935-bd55e07ac803" containerName="manager" Jul 13 21:39:24.092542 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.092520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.097305 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.097285 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fwt47\"" Jul 13 21:39:24.105102 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.105075 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs"] Jul 13 21:39:24.182705 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.182669 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.182705 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.182709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcfv\" (UniqueName: \"kubernetes.io/projected/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-kube-api-access-blcfv\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.284321 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.284288 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.284321 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.284326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcfv\" (UniqueName: \"kubernetes.io/projected/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-kube-api-access-blcfv\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.284665 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.284644 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.295089 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.295068 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcfv\" (UniqueName: \"kubernetes.io/projected/471a5897-ee37-48a7-9517-f5f2a3d7f1d0-kube-api-access-blcfv\") pod \"kuadrant-operator-controller-manager-86878c769f-7qnxs\" (UID: \"471a5897-ee37-48a7-9517-f5f2a3d7f1d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.403438 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.403344 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:24.547805 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:24.547777 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs"] Jul 13 21:39:24.549473 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:39:24.549446 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471a5897_ee37_48a7_9517_f5f2a3d7f1d0.slice/crio-7f33ec40b55e667ed3ec1d27c59067fe1c487a092538dd17634325d88cdf808c WatchSource:0}: Error finding container 7f33ec40b55e667ed3ec1d27c59067fe1c487a092538dd17634325d88cdf808c: Status 404 returned error can't find the container with id 7f33ec40b55e667ed3ec1d27c59067fe1c487a092538dd17634325d88cdf808c Jul 13 21:39:25.129175 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:25.129104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" event={"ID":"471a5897-ee37-48a7-9517-f5f2a3d7f1d0","Type":"ContainerStarted","Data":"a0e1ea466287814d4f1f6955283fa2c82cdf147ed79b0241b153b60443fa822c"} Jul 13 21:39:25.129550 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:25.129191 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" event={"ID":"471a5897-ee37-48a7-9517-f5f2a3d7f1d0","Type":"ContainerStarted","Data":"7f33ec40b55e667ed3ec1d27c59067fe1c487a092538dd17634325d88cdf808c"} Jul 13 21:39:25.129550 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:25.129345 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:39:25.150935 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:25.150878 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" podStartSLOduration=1.150862664 podStartE2EDuration="1.150862664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:39:25.148352168 +0000 UTC m=+1041.202886755" watchObservedRunningTime="2026-07-13 21:39:25.150862664 +0000 UTC m=+1041.205397251" Jul 13 21:39:36.137321 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:39:36.137235 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7qnxs" Jul 13 21:40:58.305249 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:40:58.305217 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:40:59.834961 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:40:59.834931 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:41:02.409975 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:41:02.409939 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:41:02.507825 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:41:02.507787 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:42:22.822580 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:42:22.822538 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:43:00.516994 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:43:00.516904 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:00.155465 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.155428 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs"] Jul 13 21:45:00.158842 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.158823 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" Jul 13 21:45:00.161624 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.161601 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:45:00.168255 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.168239 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" Jul 13 21:45:00.224073 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.224041 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs"] Jul 13 21:45:00.516297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.513289 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs"] Jul 13 21:45:00.522484 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:45:00.522447 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd924d256_abc1_4b83_9e3a_ba482cbe84eb.slice/crio-452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1 WatchSource:0}: Error finding container 452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1: Status 404 returned error can't find the container with id 452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1 Jul 13 21:45:00.525170 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:00.525151 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 21:45:01.030074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:01.030035 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" event={"ID":"d924d256-abc1-4b83-9e3a-ba482cbe84eb","Type":"ContainerStarted","Data":"452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1"} Jul 13 21:45:08.074536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:08.074495 2564 generic.go:356] "Generic (PLEG): container finished" podID="d924d256-abc1-4b83-9e3a-ba482cbe84eb" containerID="1acf32bf23f87cfa5324ff4503fa89572abbc1ebb7e861cf1cbeaf806610079d" exitCode=0 Jul 13 21:45:08.074926 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:08.074559 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" event={"ID":"d924d256-abc1-4b83-9e3a-ba482cbe84eb","Type":"ContainerDied","Data":"1acf32bf23f87cfa5324ff4503fa89572abbc1ebb7e861cf1cbeaf806610079d"} Jul 13 21:45:09.225585 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:09.225561 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" Jul 13 21:45:09.405060 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:09.404967 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:10.087871 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:10.087843 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" Jul 13 21:45:10.088028 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:10.087882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-qb5vs" event={"ID":"d924d256-abc1-4b83-9e3a-ba482cbe84eb","Type":"ContainerDied","Data":"452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1"} Jul 13 21:45:10.088028 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:10.087930 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="452f932bf1e15eb5e39e0358fd1337de005d5769eb6664f490949df7c5c311a1" Jul 13 21:45:18.871024 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:18.870990 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:27.140308 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:27.140259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:32.284970 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:32.284928 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:35.155167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:35.155105 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:35.247578 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:35.247545 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:35.602079 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:35.602041 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:41.351613 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:41.351559 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:41.450852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:41.450818 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:41.561611 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:41.561578 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:41.619927 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:41.619843 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:42.739190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.739108 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:42.844250 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.844217 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:45:42.844716 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.844700 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d924d256-abc1-4b83-9e3a-ba482cbe84eb" containerName="cleanup" Jul 13 21:45:42.844778 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.844718 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d924d256-abc1-4b83-9e3a-ba482cbe84eb" containerName="cleanup" Jul 13 21:45:42.844834 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.844821 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d924d256-abc1-4b83-9e3a-ba482cbe84eb" containerName="cleanup" Jul 13 21:45:42.846431 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.846383 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:42.852766 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.852749 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:42.856201 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.856170 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9b188814-serving-cert\"" Jul 13 21:45:42.859818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.859756 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:45:42.946441 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.946402 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:42.982485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.982451 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:42.982653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.982500 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:42.982653 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:42.982617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz78v\" (UniqueName: \"kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.083529 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.083490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz78v\" (UniqueName: \"kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.083719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.083558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.083719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.083589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.083719 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:45:43.083674 2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-serving-cert: secret "maas-api-e2e-ait-9b188814-serving-cert" not found Jul 13 21:45:43.083827 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:45:43.083760 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls podName:6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51 nodeName:}" failed. No retries permitted until 2026-07-13 21:45:43.583738956 +0000 UTC m=+1419.638273526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls") pod "maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" (UID: "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51") : secret "maas-api-e2e-ait-9b188814-serving-cert" not found Jul 13 21:45:43.084187 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.084168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.092600 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.092581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz78v\" (UniqueName: \"kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.307274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.307238 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:43.485952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.485842 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:43.588299 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.588260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.590757 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.590733 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") pod \"maas-api-e2e-ait-9b188814-575d46bfc-pxqx7\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.653338 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.653301 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:43.766502 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.766474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:43.921536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:43.921504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:45:43.923425 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:45:43.923390 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6803c4db_eb30_4ad0_a48a_ccf1bb7bfc51.slice/crio-d9fece33d3703980a492325964a38dac74d1406e366eb9a694f151d47861cc76 WatchSource:0}: Error finding container d9fece33d3703980a492325964a38dac74d1406e366eb9a694f151d47861cc76: Status 404 returned error can't find the container with id d9fece33d3703980a492325964a38dac74d1406e366eb9a694f151d47861cc76 Jul 13 21:45:44.293704 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:44.293670 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" event={"ID":"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51","Type":"ContainerStarted","Data":"d9fece33d3703980a492325964a38dac74d1406e366eb9a694f151d47861cc76"} Jul 13 21:45:45.301421 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:45.301385 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:45:45.302093 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:45.302062 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" event={"ID":"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51","Type":"ContainerStarted","Data":"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9"} Jul 13 21:45:45.302248 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:45.302161 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:45.329963 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:45.329911 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" podStartSLOduration=2.904087231 podStartE2EDuration="3.329895998s" totalImagesPullingTime="425.808767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:45.326813684 +0000 UTC m=+1421.381348275" watchObservedRunningTime="2026-07-13 21:45:45.329895998 +0000 UTC m=+1421.384430632" Jul 13 21:45:47.725732 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.725687 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:45:47.730778 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.730742 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:45:47.733698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.733666 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:45:47.733872 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.733728 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:45:47.834196 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.834161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b188814-vx2k2\" (UID: \"de66ef7f-d80b-44ba-b86d-597e7422ba67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:45:47.934818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.934787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b188814-vx2k2\" (UID: \"de66ef7f-d80b-44ba-b86d-597e7422ba67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:45:47.935502 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:47.935482 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b188814-vx2k2\" (UID: \"de66ef7f-d80b-44ba-b86d-597e7422ba67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:45:48.042274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:48.042239 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:45:48.185010 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:48.184983 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:45:48.186483 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:45:48.186454 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde66ef7f_d80b_44ba_b86d_597e7422ba67.slice/crio-e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a WatchSource:0}: Error finding container e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a: Status 404 returned error can't find the container with id e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a Jul 13 21:45:48.320679 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:48.320570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" event={"ID":"de66ef7f-d80b-44ba-b86d-597e7422ba67","Type":"ContainerStarted","Data":"e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a"} Jul 13 21:45:49.328220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:49.328183 2564 generic.go:356] "Generic (PLEG): container finished" podID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerID="f0df46e3d34112bafe624fad84af0be54b6b87976f46ee12eb348fcb8d8b9cbb" exitCode=7 Jul 13 21:45:49.328699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:49.328244 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" event={"ID":"de66ef7f-d80b-44ba-b86d-597e7422ba67","Type":"ContainerDied","Data":"f0df46e3d34112bafe624fad84af0be54b6b87976f46ee12eb348fcb8d8b9cbb"} Jul 13 21:45:49.328699 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:49.328496 2564 scope.go:117] "RemoveContainer" containerID="f0df46e3d34112bafe624fad84af0be54b6b87976f46ee12eb348fcb8d8b9cbb" Jul 13 21:45:50.336182 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:50.336143 2564 generic.go:356] "Generic (PLEG): container finished" podID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerID="7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521" exitCode=7 Jul 13 21:45:50.336830 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:50.336226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" event={"ID":"de66ef7f-d80b-44ba-b86d-597e7422ba67","Type":"ContainerDied","Data":"7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521"} Jul 13 21:45:50.336830 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:50.336292 2564 scope.go:117] "RemoveContainer" containerID="f0df46e3d34112bafe624fad84af0be54b6b87976f46ee12eb348fcb8d8b9cbb" Jul 13 21:45:50.336830 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:50.336371 2564 scope.go:117] "RemoveContainer" containerID="7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521" Jul 13 21:45:50.336830 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:45:50.336612 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9b188814-vx2k2_odh-ai-gateway-infra(de66ef7f-d80b-44ba-b86d-597e7422ba67)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" Jul 13 21:45:51.313348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:51.313319 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:45:51.344647 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:45:51.344619 2564 scope.go:117] "RemoveContainer" containerID="7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521" Jul 13 21:45:51.345019 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:45:51.344827 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9b188814-vx2k2_odh-ai-gateway-infra(de66ef7f-d80b-44ba-b86d-597e7422ba67)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" Jul 13 21:46:05.522890 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:05.522806 2564 scope.go:117] "RemoveContainer" containerID="7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521" Jul 13 21:46:06.429574 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:06.429543 2564 generic.go:356] "Generic (PLEG): container finished" podID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerID="0eafea0a31bfc58e988a5971e3864938881654c09f3d06b22bdd69925ade9d26" exitCode=0 Jul 13 21:46:06.429761 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:06.429624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" event={"ID":"de66ef7f-d80b-44ba-b86d-597e7422ba67","Type":"ContainerDied","Data":"0eafea0a31bfc58e988a5971e3864938881654c09f3d06b22bdd69925ade9d26"} Jul 13 21:46:06.429761 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:06.429692 2564 scope.go:117] "RemoveContainer" containerID="7e4695558a7ada74e918c03c17fa3259df4cbe1e7c32671c636942899bb90521" Jul 13 21:46:07.462147 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:07.462102 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:46:07.580843 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:07.580818 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:46:07.735214 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:07.735106 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca\") pod \"de66ef7f-d80b-44ba-b86d-597e7422ba67\" (UID: \"de66ef7f-d80b-44ba-b86d-597e7422ba67\") " Jul 13 21:46:07.735467 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:07.735443 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de66ef7f-d80b-44ba-b86d-597e7422ba67" (UID: "de66ef7f-d80b-44ba-b86d-597e7422ba67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:46:07.836377 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:07.836335 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de66ef7f-d80b-44ba-b86d-597e7422ba67-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:46:08.443965 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.443931 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" Jul 13 21:46:08.444162 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.443932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2" event={"ID":"de66ef7f-d80b-44ba-b86d-597e7422ba67","Type":"ContainerDied","Data":"e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a"} Jul 13 21:46:08.444162 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.444055 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e89ed7601fc407e417515399c39caab4dd70b9fb34035abe2c7a549d7fba191a" Jul 13 21:46:08.475068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.475033 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:46:08.478329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.478302 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b188814-vx2k2"] Jul 13 21:46:08.529520 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:08.529487 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" path="/var/lib/kubelet/pods/de66ef7f-d80b-44ba-b86d-597e7422ba67/volumes" Jul 13 21:46:58.441760 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.441723 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:46:58.442407 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.441981 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" podUID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" containerName="maas-api" containerID="cri-o://7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9" gracePeriod=30 Jul 13 21:46:58.695485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.695401 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:46:58.730927 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.730885 2564 generic.go:356] "Generic (PLEG): container finished" podID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" containerID="7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9" exitCode=0 Jul 13 21:46:58.730927 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.730932 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" Jul 13 21:46:58.731221 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.730963 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" event={"ID":"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51","Type":"ContainerDied","Data":"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9"} Jul 13 21:46:58.731221 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.731021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7" event={"ID":"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51","Type":"ContainerDied","Data":"d9fece33d3703980a492325964a38dac74d1406e366eb9a694f151d47861cc76"} Jul 13 21:46:58.731221 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.731046 2564 scope.go:117] "RemoveContainer" containerID="7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9" Jul 13 21:46:58.742239 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.742219 2564 scope.go:117] "RemoveContainer" containerID="7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9" Jul 13 21:46:58.742519 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:46:58.742497 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9\": container with ID starting with 7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9 not found: ID does not exist" containerID="7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9" Jul 13 21:46:58.742580 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.742530 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9"} err="failed to get container status \"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9\": rpc error: code = NotFound desc = could not find container \"7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9\": container with ID starting with 7b16bce734bc2b51a6e0af522ab488e48db6dd9c906c522e473aecbffd00dfd9 not found: ID does not exist" Jul 13 21:46:58.811190 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.811154 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca\") pod \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " Jul 13 21:46:58.811337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.811202 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") pod \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " Jul 13 21:46:58.811337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.811286 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz78v\" (UniqueName: \"kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v\") pod \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\" (UID: \"6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51\") " Jul 13 21:46:58.811611 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.811584 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" (UID: "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:46:58.813508 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.813478 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v" (OuterVolumeSpecName: "kube-api-access-cz78v") pod "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" (UID: "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51"). InnerVolumeSpecName "kube-api-access-cz78v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:46:58.813612 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.813515 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" (UID: "6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:46:58.911922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.911893 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-openshift-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:46:58.911922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.911917 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-maas-api-tls\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:46:58.911922 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:58.911926 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz78v\" (UniqueName: \"kubernetes.io/projected/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51-kube-api-access-cz78v\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:46:59.054909 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:59.054877 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:46:59.061644 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:59.061609 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9b188814-575d46bfc-pxqx7"] Jul 13 21:46:59.198593 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:46:59.198554 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:00.530554 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:00.530516 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" path="/var/lib/kubelet/pods/6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51/volumes" Jul 13 21:47:02.906769 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:02.906732 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:08.764978 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:08.764940 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:08.834663 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:08.834632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:13.103278 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:13.103232 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:13.458572 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:13.458474 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:13.578059 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:13.578029 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:13.828961 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:13.828919 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.062181 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.062143 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.139492 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.139458 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:47:15.140069 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140054 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140073 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140087 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" containerName="maas-api" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140096 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" containerName="maas-api" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140107 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140133 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140147 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140156 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140265 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.140504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140280 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6803c4db-eb30-4ad0-a48a-ccf1bb7bfc51" containerName="maas-api" Jul 13 21:47:15.140504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.140291 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:47:15.145016 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.144994 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.147423 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.147403 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c9a3e9a3-serving-cert\"" Jul 13 21:47:15.152654 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.152628 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:47:15.171530 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.171503 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.262927 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.262894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.263154 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.262958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.263154 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.263073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngw44\" (UniqueName: \"kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.334196 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.334074 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.364302 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.364263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngw44\" (UniqueName: \"kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.364473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.364322 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.364473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.364374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.365014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.364988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.367091 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.367071 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.372478 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.372457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngw44\" (UniqueName: \"kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44\") pod \"maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.457274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.457238 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:15.603436 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.603403 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:47:15.604301 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:47:15.604272 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e0d5ce_77ce_4f5c_a768_7f83265ffb5c.slice/crio-cc51bd28e0ad8e2cab6a94bd21b00cd15541626dca4ef9b74a2b872c2607c23d WatchSource:0}: Error finding container cc51bd28e0ad8e2cab6a94bd21b00cd15541626dca4ef9b74a2b872c2607c23d: Status 404 returned error can't find the container with id cc51bd28e0ad8e2cab6a94bd21b00cd15541626dca4ef9b74a2b872c2607c23d Jul 13 21:47:15.681704 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.681673 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.782923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.782890 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:15.832439 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:15.832406 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" event={"ID":"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c","Type":"ContainerStarted","Data":"cc51bd28e0ad8e2cab6a94bd21b00cd15541626dca4ef9b74a2b872c2607c23d"} Jul 13 21:47:16.841391 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:16.841358 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" event={"ID":"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c","Type":"ContainerStarted","Data":"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de"} Jul 13 21:47:16.841763 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:16.841487 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:47:16.859927 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:16.859875 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" podStartSLOduration=1.402530243 podStartE2EDuration="1.859860998s" totalImagesPullingTime="457.330755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:47:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:47:16.858739206 +0000 UTC m=+1512.913273794" watchObservedRunningTime="2026-07-13 21:47:16.859860998 +0000 UTC m=+1512.914395592" Jul 13 21:47:19.542014 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:19.541972 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:19.610386 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:19.610336 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:22.836526 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:22.836484 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:47:22.854110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:47:22.854086 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:48:19.588285 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:19.588249 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:19.701440 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:19.701406 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:20.088520 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.088485 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:48:20.088814 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.088785 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" podUID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" containerName="maas-api" containerID="cri-o://c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de" gracePeriod=30 Jul 13 21:48:20.118084 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.118056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:20.341698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.341636 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:48:20.469619 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.469583 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca\") pod \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " Jul 13 21:48:20.469790 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.469705 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls\") pod \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " Jul 13 21:48:20.469790 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.469731 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngw44\" (UniqueName: \"kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44\") pod \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\" (UID: \"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c\") " Jul 13 21:48:20.470018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.469995 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" (UID: "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:48:20.470087 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.470075 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-openshift-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:48:20.471797 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.471764 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" (UID: "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:48:20.471945 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.471820 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44" (OuterVolumeSpecName: "kube-api-access-ngw44") pod "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" (UID: "99e0d5ce-77ce-4f5c-a768-7f83265ffb5c"). InnerVolumeSpecName "kube-api-access-ngw44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:48:20.571554 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.571516 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-maas-api-tls\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:48:20.571554 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:20.571550 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngw44\" (UniqueName: \"kubernetes.io/projected/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c-kube-api-access-ngw44\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:48:21.095297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.095265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:21.158231 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.158195 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:21.206937 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.206899 2564 generic.go:356] "Generic (PLEG): container finished" podID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" containerID="c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de" exitCode=0 Jul 13 21:48:21.207156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.206952 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" Jul 13 21:48:21.207156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.206981 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" event={"ID":"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c","Type":"ContainerDied","Data":"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de"} Jul 13 21:48:21.207156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.207034 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn" event={"ID":"99e0d5ce-77ce-4f5c-a768-7f83265ffb5c","Type":"ContainerDied","Data":"cc51bd28e0ad8e2cab6a94bd21b00cd15541626dca4ef9b74a2b872c2607c23d"} Jul 13 21:48:21.207156 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.207055 2564 scope.go:117] "RemoveContainer" containerID="c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de" Jul 13 21:48:21.216356 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.216335 2564 scope.go:117] "RemoveContainer" containerID="c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de" Jul 13 21:48:21.216628 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:48:21.216610 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de\": container with ID starting with c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de not found: ID does not exist" containerID="c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de" Jul 13 21:48:21.216686 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.216636 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de"} err="failed to get container status \"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de\": rpc error: code = NotFound desc = could not find container \"c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de\": container with ID starting with c0067fd8667986ede46ace164f8c8d635bb4c5419100fffd5f30bf06722a61de not found: ID does not exist" Jul 13 21:48:21.225198 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.225175 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:48:21.229225 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:21.229199 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c9a3e9a3-5ddd95c879-xtqdn"] Jul 13 21:48:22.529735 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:22.529694 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" path="/var/lib/kubelet/pods/99e0d5ce-77ce-4f5c-a768-7f83265ffb5c/volumes" Jul 13 21:48:25.437180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:25.437141 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:25.511519 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:25.511486 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:25.655556 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:25.655519 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:26.016713 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.016682 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:26.090608 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.090577 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:26.845704 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.845651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:26.921606 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.921573 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:48:26.921997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.921984 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" containerName="maas-api" Jul 13 21:48:26.921997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.921998 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" containerName="maas-api" Jul 13 21:48:26.922164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.922064 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e0d5ce-77ce-4f5c-a768-7f83265ffb5c" containerName="maas-api" Jul 13 21:48:26.922164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.922075 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de66ef7f-d80b-44ba-b86d-597e7422ba67" containerName="revoke-keys" Jul 13 21:48:26.926571 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.926552 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:26.929213 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.929193 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8a043b23-serving-cert\"" Jul 13 21:48:26.939729 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.939706 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:48:26.949282 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:26.949259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:27.026873 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.026836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.027048 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.026880 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.027048 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.026986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwzx\" (UniqueName: \"kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.032586 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.032556 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:27.128030 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.127933 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.128030 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.127979 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.128300 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.128058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwzx\" (UniqueName: \"kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.128687 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.128668 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.130433 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.130414 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.138627 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.138599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwzx\" (UniqueName: \"kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx\") pod \"maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.236403 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.236362 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:27.406567 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.406541 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:48:27.408351 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:48:27.408325 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa630d05_81b4_4d35_bfae_0913132ef033.slice/crio-fe3f0354d09aaf6b25fd2c9a079c3bb2b62303b2664663229e56778ebd7fa5cf WatchSource:0}: Error finding container fe3f0354d09aaf6b25fd2c9a079c3bb2b62303b2664663229e56778ebd7fa5cf: Status 404 returned error can't find the container with id fe3f0354d09aaf6b25fd2c9a079c3bb2b62303b2664663229e56778ebd7fa5cf Jul 13 21:48:27.775605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.775571 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:27.878840 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:27.878803 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:28.250141 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:28.250027 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" event={"ID":"aa630d05-81b4-4d35-bfae-0913132ef033","Type":"ContainerStarted","Data":"6360e224a99b6cf9be24cfec83e8fbcbfcb8e6df159a3357e6ed3af03378db76"} Jul 13 21:48:28.250141 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:28.250087 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" event={"ID":"aa630d05-81b4-4d35-bfae-0913132ef033","Type":"ContainerStarted","Data":"fe3f0354d09aaf6b25fd2c9a079c3bb2b62303b2664663229e56778ebd7fa5cf"} Jul 13 21:48:28.250388 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:28.250167 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:48:28.268481 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:28.268422 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" podStartSLOduration=1.865465433 podStartE2EDuration="2.268402169s" totalImagesPullingTime="402.936736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:48:28.266699678 +0000 UTC m=+1584.321234265" watchObservedRunningTime="2026-07-13 21:48:28.268402169 +0000 UTC m=+1584.322936756" Jul 13 21:48:29.462008 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:29.461977 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:48:34.261980 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:48:34.261947 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:49:02.436288 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:02.436198 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:02.535713 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:02.535678 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:22.580220 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:22.580180 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:22.681385 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:22.681338 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:32.509076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.509038 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:49:32.509501 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.509340 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" podUID="aa630d05-81b4-4d35-bfae-0913132ef033" containerName="maas-api" containerID="cri-o://6360e224a99b6cf9be24cfec83e8fbcbfcb8e6df159a3357e6ed3af03378db76" gracePeriod=30 Jul 13 21:49:32.533333 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.533248 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:32.640788 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.640665 2564 generic.go:356] "Generic (PLEG): container finished" podID="aa630d05-81b4-4d35-bfae-0913132ef033" containerID="6360e224a99b6cf9be24cfec83e8fbcbfcb8e6df159a3357e6ed3af03378db76" exitCode=0 Jul 13 21:49:32.640788 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.640728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" event={"ID":"aa630d05-81b4-4d35-bfae-0913132ef033","Type":"ContainerDied","Data":"6360e224a99b6cf9be24cfec83e8fbcbfcb8e6df159a3357e6ed3af03378db76"} Jul 13 21:49:32.768865 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.768842 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:49:32.861482 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.861451 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnwzx\" (UniqueName: \"kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx\") pod \"aa630d05-81b4-4d35-bfae-0913132ef033\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " Jul 13 21:49:32.861648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.861555 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca\") pod \"aa630d05-81b4-4d35-bfae-0913132ef033\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " Jul 13 21:49:32.861648 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.861607 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls\") pod \"aa630d05-81b4-4d35-bfae-0913132ef033\" (UID: \"aa630d05-81b4-4d35-bfae-0913132ef033\") " Jul 13 21:49:32.861901 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.861875 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa630d05-81b4-4d35-bfae-0913132ef033" (UID: "aa630d05-81b4-4d35-bfae-0913132ef033"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:49:32.863636 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.863611 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx" (OuterVolumeSpecName: "kube-api-access-fnwzx") pod "aa630d05-81b4-4d35-bfae-0913132ef033" (UID: "aa630d05-81b4-4d35-bfae-0913132ef033"). InnerVolumeSpecName "kube-api-access-fnwzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:49:32.863758 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.863742 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa630d05-81b4-4d35-bfae-0913132ef033" (UID: "aa630d05-81b4-4d35-bfae-0913132ef033"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:49:32.962466 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.962428 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa630d05-81b4-4d35-bfae-0913132ef033-maas-api-tls\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:49:32.962466 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.962462 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnwzx\" (UniqueName: \"kubernetes.io/projected/aa630d05-81b4-4d35-bfae-0913132ef033-kube-api-access-fnwzx\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:49:32.962466 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:32.962473 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa630d05-81b4-4d35-bfae-0913132ef033-openshift-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:49:33.647876 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:33.647847 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" Jul 13 21:49:33.648416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:33.647887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r" event={"ID":"aa630d05-81b4-4d35-bfae-0913132ef033","Type":"ContainerDied","Data":"fe3f0354d09aaf6b25fd2c9a079c3bb2b62303b2664663229e56778ebd7fa5cf"} Jul 13 21:49:33.648416 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:33.647944 2564 scope.go:117] "RemoveContainer" containerID="6360e224a99b6cf9be24cfec83e8fbcbfcb8e6df159a3357e6ed3af03378db76" Jul 13 21:49:33.677794 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:33.676188 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:49:33.683527 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:33.683494 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a043b23-67bf64db47-pbc5r"] Jul 13 21:49:34.532098 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:34.532063 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa630d05-81b4-4d35-bfae-0913132ef033" path="/var/lib/kubelet/pods/aa630d05-81b4-4d35-bfae-0913132ef033/volumes" Jul 13 21:49:36.935328 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:36.935290 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:42.974678 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:42.974640 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:43.050492 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:43.050460 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:47.950310 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:47.950265 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:48.226821 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:48.226720 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:49.415873 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.415831 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:49.552960 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.552927 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:49:49.553392 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.553379 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa630d05-81b4-4d35-bfae-0913132ef033" containerName="maas-api" Jul 13 21:49:49.553445 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.553394 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa630d05-81b4-4d35-bfae-0913132ef033" containerName="maas-api" Jul 13 21:49:49.553485 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.553460 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa630d05-81b4-4d35-bfae-0913132ef033" containerName="maas-api" Jul 13 21:49:49.556626 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.556602 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.559691 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.559668 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7fdae48e-serving-cert\"" Jul 13 21:49:49.569825 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.569798 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:49.580273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.580242 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:49:49.624399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.624366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.624570 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.624404 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.624570 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.624543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5d2l\" (UniqueName: \"kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.701695 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.701605 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:49.725754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.725714 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5d2l\" (UniqueName: \"kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.725925 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.725796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.725925 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.725832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.726517 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.726490 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.728560 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.728536 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.737923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.737903 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5d2l\" (UniqueName: \"kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l\") pod \"maas-api-e2e-derive-7fdae48e-5666996647-bttbw\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:49.775045 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.775015 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:49.867264 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:49.867221 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:50.027483 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.027450 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:49:50.028606 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:49:50.028574 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa54895_4c1c_48fe_a135_f2bf78fa3181.slice/crio-8eccf60cdd14b0a91e4c9381edff62a49b4dcd539d0af9183bc10f18a72e5be4 WatchSource:0}: Error finding container 8eccf60cdd14b0a91e4c9381edff62a49b4dcd539d0af9183bc10f18a72e5be4: Status 404 returned error can't find the container with id 8eccf60cdd14b0a91e4c9381edff62a49b4dcd539d0af9183bc10f18a72e5be4 Jul 13 21:49:50.285100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.285066 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:50.384921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.384888 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:50.757548 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.757459 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" event={"ID":"4aa54895-4c1c-48fe-a135-f2bf78fa3181","Type":"ContainerStarted","Data":"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3"} Jul 13 21:49:50.757548 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.757525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" event={"ID":"4aa54895-4c1c-48fe-a135-f2bf78fa3181","Type":"ContainerStarted","Data":"8eccf60cdd14b0a91e4c9381edff62a49b4dcd539d0af9183bc10f18a72e5be4"} Jul 13 21:49:50.757953 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.757551 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:50.777466 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:50.777413 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" podStartSLOduration=1.361017747 podStartE2EDuration="1.777397103s" totalImagesPullingTime="416.379356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:49:50.775004667 +0000 UTC m=+1666.829539254" watchObservedRunningTime="2026-07-13 21:49:50.777397103 +0000 UTC m=+1666.831931689" Jul 13 21:49:54.410835 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:54.410798 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:54.510852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:54.510817 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:56.769131 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:56.769083 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:49:57.607233 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:57.607194 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:58.358627 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:58.358591 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:49:58.431459 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:49:58.431426 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:50:41.501467 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:50:41.501376 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:50:41.601793 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:50:41.601763 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:04.254911 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.254863 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:51:04.255502 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.255164 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" podUID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" containerName="maas-api" containerID="cri-o://5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3" gracePeriod=30 Jul 13 21:51:04.291858 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.291817 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:04.513498 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.513476 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:51:04.616996 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.616967 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5d2l\" (UniqueName: \"kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l\") pod \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " Jul 13 21:51:04.617187 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.617017 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls\") pod \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " Jul 13 21:51:04.617187 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.617143 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca\") pod \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\" (UID: \"4aa54895-4c1c-48fe-a135-f2bf78fa3181\") " Jul 13 21:51:04.617500 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.617477 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4aa54895-4c1c-48fe-a135-f2bf78fa3181" (UID: "4aa54895-4c1c-48fe-a135-f2bf78fa3181"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:51:04.619375 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.619344 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l" (OuterVolumeSpecName: "kube-api-access-p5d2l") pod "4aa54895-4c1c-48fe-a135-f2bf78fa3181" (UID: "4aa54895-4c1c-48fe-a135-f2bf78fa3181"). InnerVolumeSpecName "kube-api-access-p5d2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 21:51:04.619375 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.619366 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4aa54895-4c1c-48fe-a135-f2bf78fa3181" (UID: "4aa54895-4c1c-48fe-a135-f2bf78fa3181"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 21:51:04.718901 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.718865 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa54895-4c1c-48fe-a135-f2bf78fa3181-maas-api-tls\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:51:04.718901 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.718902 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa54895-4c1c-48fe-a135-f2bf78fa3181-openshift-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:51:04.719150 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:04.718917 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5d2l\" (UniqueName: \"kubernetes.io/projected/4aa54895-4c1c-48fe-a135-f2bf78fa3181-kube-api-access-p5d2l\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:51:05.191931 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.191896 2564 generic.go:356] "Generic (PLEG): container finished" podID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" containerID="5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3" exitCode=0 Jul 13 21:51:05.192110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.191956 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" Jul 13 21:51:05.192110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.191982 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" event={"ID":"4aa54895-4c1c-48fe-a135-f2bf78fa3181","Type":"ContainerDied","Data":"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3"} Jul 13 21:51:05.192110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.192054 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw" event={"ID":"4aa54895-4c1c-48fe-a135-f2bf78fa3181","Type":"ContainerDied","Data":"8eccf60cdd14b0a91e4c9381edff62a49b4dcd539d0af9183bc10f18a72e5be4"} Jul 13 21:51:05.192110 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.192076 2564 scope.go:117] "RemoveContainer" containerID="5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3" Jul 13 21:51:05.201720 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.201704 2564 scope.go:117] "RemoveContainer" containerID="5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3" Jul 13 21:51:05.201997 ip-10-0-131-109 kubenswrapper[2564]: E0713 21:51:05.201979 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3\": container with ID starting with 5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3 not found: ID does not exist" containerID="5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3" Jul 13 21:51:05.202076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.202010 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3"} err="failed to get container status \"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3\": rpc error: code = NotFound desc = could not find container \"5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3\": container with ID starting with 5406e3a514b6cb80e2a2b9d0a568d796c93e2f02f435ce08c20d5c8276eee3d3 not found: ID does not exist" Jul 13 21:51:05.214024 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.213994 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:51:05.218348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:05.218324 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7fdae48e-5666996647-bttbw"] Jul 13 21:51:06.529418 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:06.529379 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" path="/var/lib/kubelet/pods/4aa54895-4c1c-48fe-a135-f2bf78fa3181/volumes" Jul 13 21:51:08.876172 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:08.876112 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:11.523335 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:11.523299 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:14.805036 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:14.804998 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:14.909969 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:14.909915 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:19.068680 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:19.068646 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:19.633563 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:19.633519 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:19.733147 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:19.733104 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:19.832970 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:19.832932 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:21.215453 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:21.215413 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:25.723802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:25.723763 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:27.218562 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:27.218525 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:27.430723 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:27.430679 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:27.558301 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:27.558260 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:27.943006 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:27.942919 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:51:53.846800 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:51:53.846766 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:01.517270 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:01.517157 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:02.736605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:02.736573 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:17.668350 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:17.668314 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:23.488715 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:23.488674 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:23.600623 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:23.600580 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:23.703347 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:23.703303 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:34.177955 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:34.177907 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:34.945851 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:34.945815 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:40.574111 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:40.574081 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:40.679006 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:40.678973 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:40.776858 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:40.776813 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:40.884656 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:40.884567 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:41.634092 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:41.633992 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:41.735011 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:41.734976 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:41.830677 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:41.830635 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:43.348019 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:43.347977 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:49.154564 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:49.154522 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:49.256007 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:49.255956 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:49.803068 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:49.803030 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:55.572813 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:55.572775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:55.673532 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:55.673497 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:55.814164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:55.814131 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:56.105274 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:56.105241 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:56.419602 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:56.419506 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:58.751454 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:58.751410 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:58.853823 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:58.853790 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:59.115079 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:59.115046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:59.185085 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:59.185052 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:59.578033 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:59.577997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:52:59.681085 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:52:59.681049 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:03.217812 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:03.217776 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:03.847461 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:03.847414 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:03.951938 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:03.951898 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:04.255691 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:04.255598 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:04.637349 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:04.637317 2564 scope.go:117] "RemoveContainer" containerID="0eafea0a31bfc58e988a5971e3864938881654c09f3d06b22bdd69925ade9d26" Jul 13 21:53:09.860911 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:09.860860 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:09.960908 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:09.960874 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:20.608104 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:20.608061 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:25.295978 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:25.295920 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:31.252167 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:31.252059 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:31.339018 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:31.338983 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:35.919315 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:35.919278 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:36.129750 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:36.129711 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:36.249864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:36.249761 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:36.349329 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:36.349295 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:36.452828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:36.452793 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:38.166533 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:38.166500 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:38.262719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:38.262682 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:42.070136 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:42.070084 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:42.140046 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:42.140006 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:42.222242 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:42.222209 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:42.401456 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:42.401356 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:48.777578 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:48.777540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:48.875360 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:48.875320 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:55.489206 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:55.489161 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:55.828982 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:55.828943 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:55.980923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:55.980882 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:58.924437 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:58.924399 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:59.042112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:59.042074 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:59.176706 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:59.176625 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:59.554583 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:59.554545 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:59.779495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:59.779454 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:53:59.946651 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:53:59.946562 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:03.488861 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:03.488825 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:03.679745 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:03.679689 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:03.822136 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:03.822086 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:10.165719 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:10.165679 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:21.216460 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:21.216428 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:21.396675 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:21.396633 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:36.629725 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.629687 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4"] Jul 13 21:54:36.630173 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.630162 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" containerName="maas-api" Jul 13 21:54:36.630173 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.630175 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" containerName="maas-api" Jul 13 21:54:36.630277 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.630234 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aa54895-4c1c-48fe-a135-f2bf78fa3181" containerName="maas-api" Jul 13 21:54:36.633464 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.633445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:36.642197 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.642180 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:54:36.653900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.653875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4"] Jul 13 21:54:36.705783 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.705742 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bfec26b9-27df4\" (UID: \"35b4152f-eadc-4dbf-80d6-da60c8dde2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:36.806845 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.806807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bfec26b9-27df4\" (UID: \"35b4152f-eadc-4dbf-80d6-da60c8dde2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:36.807448 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.807430 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bfec26b9-27df4\" (UID: \"35b4152f-eadc-4dbf-80d6-da60c8dde2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:36.943900 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:36.943817 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:37.091705 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:37.091676 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4"] Jul 13 21:54:37.092427 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:54:37.092399 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b4152f_eadc_4dbf_80d6_da60c8dde2e4.slice/crio-e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31 WatchSource:0}: Error finding container e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31: Status 404 returned error can't find the container with id e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31 Jul 13 21:54:37.432468 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:37.432434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" event={"ID":"35b4152f-eadc-4dbf-80d6-da60c8dde2e4","Type":"ContainerStarted","Data":"e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31"} Jul 13 21:54:38.439831 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:38.439735 2564 generic.go:356] "Generic (PLEG): container finished" podID="35b4152f-eadc-4dbf-80d6-da60c8dde2e4" containerID="d5699322a88e3f1cb25397338efc35b969cdb61280cdd41e28ea3394fec36fa2" exitCode=0 Jul 13 21:54:38.440273 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:38.439823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" event={"ID":"35b4152f-eadc-4dbf-80d6-da60c8dde2e4","Type":"ContainerDied","Data":"d5699322a88e3f1cb25397338efc35b969cdb61280cdd41e28ea3394fec36fa2"} Jul 13 21:54:39.583875 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:39.583853 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:39.734850 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:39.734765 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca\") pod \"35b4152f-eadc-4dbf-80d6-da60c8dde2e4\" (UID: \"35b4152f-eadc-4dbf-80d6-da60c8dde2e4\") " Jul 13 21:54:39.735127 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:39.735092 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "35b4152f-eadc-4dbf-80d6-da60c8dde2e4" (UID: "35b4152f-eadc-4dbf-80d6-da60c8dde2e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:54:39.835753 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:39.835718 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35b4152f-eadc-4dbf-80d6-da60c8dde2e4-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:54:40.452103 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:40.452076 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" Jul 13 21:54:40.452324 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:40.452070 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bfec26b9-27df4" event={"ID":"35b4152f-eadc-4dbf-80d6-da60c8dde2e4","Type":"ContainerDied","Data":"e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31"} Jul 13 21:54:40.452324 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:40.452230 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e03b705ca4d82355b32129c94fe4d0596e7e452e4355231955ddb607b3ef6e31" Jul 13 21:54:48.218177 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:48.218112 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:51.919049 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:51.919008 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:54:56.978692 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:54:56.978592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:34.036935 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:34.036895 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:34.207112 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:34.207077 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:39.735164 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:39.735099 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:39.836631 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:39.836584 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:43.994192 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:43.994154 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:44.204249 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:44.204198 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:45.210696 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:45.210652 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:45.325889 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:45.325852 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:45.559982 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:45.559947 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:46.183772 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:46.183704 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:46.281664 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:46.281632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:46.382192 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:46.382162 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:47.749045 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:47.749014 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:53.725536 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:53.725501 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:53.825061 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:53.825027 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:55:54.383046 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:55:54.383013 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:00.348864 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:00.348804 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:00.432322 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:00.432284 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:00.516362 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:00.516328 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:06.988003 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:06.987962 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:11.045575 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:11.045533 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:11.364540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:11.364457 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:32.298693 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:32.298598 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:38.106162 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:38.106127 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:38.207337 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:38.207282 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:38.435473 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:38.435369 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:38.730277 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:38.730179 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:38.830044 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:38.829997 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:41.735264 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:41.735224 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:41.827184 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:41.826463 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:41.899526 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:41.899493 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:42.188063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:42.188032 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:42.311596 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:42.311556 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:42.416507 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:42.416468 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:46.230765 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:46.230726 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:46.549852 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:46.549816 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:46.847720 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:46.847637 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:47.881774 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:47.881741 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:49.750240 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:49.750194 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:49.847785 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:49.847750 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:49.921063 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:49.921023 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:50.411134 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:50.411091 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:50.536265 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:50.536205 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:50.673921 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:50.673816 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:56:54.232472 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:56:54.232440 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:03.223365 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:03.223315 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:04.415462 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:04.415401 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:06.841527 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:06.841489 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:11.218615 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:11.218580 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:11.317358 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:11.317317 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:19.383192 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:19.383148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:33.218399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:33.218357 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:52.510835 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:52.510799 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:53.428855 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.428820 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:53.522130 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.522079 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj"] Jul 13 21:57:53.522607 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.522536 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b4152f-eadc-4dbf-80d6-da60c8dde2e4" containerName="revoke-keys" Jul 13 21:57:53.522607 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.522552 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b4152f-eadc-4dbf-80d6-da60c8dde2e4" containerName="revoke-keys" Jul 13 21:57:53.522727 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.522638 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="35b4152f-eadc-4dbf-80d6-da60c8dde2e4" containerName="revoke-keys" Jul 13 21:57:53.525828 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.525801 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:53.528476 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.528452 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:57:53.533415 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.533389 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj"] Jul 13 21:57:53.607080 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.607028 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj\" (UID: \"9f13fe94-0d16-478d-b374-e1f4ea23ad93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:53.708744 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.708642 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj\" (UID: \"9f13fe94-0d16-478d-b374-e1f4ea23ad93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:53.709359 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.709335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj\" (UID: \"9f13fe94-0d16-478d-b374-e1f4ea23ad93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:53.841611 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.841579 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:53.988836 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:53.988690 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj"] Jul 13 21:57:53.994068 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:57:53.994042 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f13fe94_0d16_478d_b374_e1f4ea23ad93.slice/crio-5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9 WatchSource:0}: Error finding container 5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9: Status 404 returned error can't find the container with id 5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9 Jul 13 21:57:54.130553 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:54.130515 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:54.232479 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:54.232446 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:57:54.579773 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:54.579735 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" event={"ID":"9f13fe94-0d16-478d-b374-e1f4ea23ad93","Type":"ContainerStarted","Data":"cc303f77769b8e05ca418cbae76dfb631ae445c0ad8865085dd4802f4c70d8a0"} Jul 13 21:57:54.580172 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:54.579787 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" event={"ID":"9f13fe94-0d16-478d-b374-e1f4ea23ad93","Type":"ContainerStarted","Data":"5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9"} Jul 13 21:57:54.596133 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:54.596075 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" podStartSLOduration=1.5960602449999999 podStartE2EDuration="1.596060245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:57:54.593151605 +0000 UTC m=+2150.647686186" watchObservedRunningTime="2026-07-13 21:57:54.596060245 +0000 UTC m=+2150.650594831" Jul 13 21:57:55.586468 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:55.586432 2564 generic.go:356] "Generic (PLEG): container finished" podID="9f13fe94-0d16-478d-b374-e1f4ea23ad93" containerID="cc303f77769b8e05ca418cbae76dfb631ae445c0ad8865085dd4802f4c70d8a0" exitCode=0 Jul 13 21:57:55.586914 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:55.586516 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" event={"ID":"9f13fe94-0d16-478d-b374-e1f4ea23ad93","Type":"ContainerDied","Data":"cc303f77769b8e05ca418cbae76dfb631ae445c0ad8865085dd4802f4c70d8a0"} Jul 13 21:57:56.733504 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:56.733480 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:56.838540 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:56.838512 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca\") pod \"9f13fe94-0d16-478d-b374-e1f4ea23ad93\" (UID: \"9f13fe94-0d16-478d-b374-e1f4ea23ad93\") " Jul 13 21:57:56.838892 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:56.838871 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f13fe94-0d16-478d-b374-e1f4ea23ad93" (UID: "9f13fe94-0d16-478d-b374-e1f4ea23ad93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:57:56.939846 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:56.939757 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f13fe94-0d16-478d-b374-e1f4ea23ad93-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:57:57.599419 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:57.599384 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" Jul 13 21:57:57.599605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:57.599383 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj" event={"ID":"9f13fe94-0d16-478d-b374-e1f4ea23ad93","Type":"ContainerDied","Data":"5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9"} Jul 13 21:57:57.599605 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:57:57.599527 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5cd53f8b96034bb66f76271abca0f340ee932f44d00d20d09928cbc9b6f7b1d9" Jul 13 21:58:03.954035 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:03.953998 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:08.669170 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:08.669132 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:14.644444 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:14.644403 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:14.706987 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:14.706949 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:19.275174 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.275136 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:19.497552 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.497522 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:19.602388 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.602299 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9"] Jul 13 21:58:19.602933 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.602914 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f13fe94-0d16-478d-b374-e1f4ea23ad93" containerName="revoke-keys" Jul 13 21:58:19.602979 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.602938 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f13fe94-0d16-478d-b374-e1f4ea23ad93" containerName="revoke-keys" Jul 13 21:58:19.603076 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.603065 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f13fe94-0d16-478d-b374-e1f4ea23ad93" containerName="revoke-keys" Jul 13 21:58:19.606283 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.606266 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:19.608991 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.608967 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:58:19.610988 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.610944 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9"] Jul 13 21:58:19.643170 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.643139 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04c954-65zv9\" (UID: \"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:19.743899 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.743870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04c954-65zv9\" (UID: \"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:19.744541 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.744522 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04c954-65zv9\" (UID: \"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:19.919292 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:19.919203 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:20.064569 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.064493 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9"] Jul 13 21:58:20.066959 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:58:20.066933 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78f4cb7_4651_41f2_9e69_d109ebfdd3cc.slice/crio-36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9 WatchSource:0}: Error finding container 36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9: Status 404 returned error can't find the container with id 36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9 Jul 13 21:58:20.215276 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.215171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:20.318528 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.318467 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:20.734930 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.734889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" event={"ID":"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc","Type":"ContainerStarted","Data":"b1fc71e0134b3e7ad88797e54a48afd62a77e4b0d6dfceddfa60e7f8256e1d88"} Jul 13 21:58:20.735141 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.734953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" event={"ID":"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc","Type":"ContainerStarted","Data":"36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9"} Jul 13 21:58:20.751148 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:20.751088 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" podStartSLOduration=1.751073916 podStartE2EDuration="1.751073916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:20.748332321 +0000 UTC m=+2176.802866909" watchObservedRunningTime="2026-07-13 21:58:20.751073916 +0000 UTC m=+2176.805608537" Jul 13 21:58:21.742257 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:21.742223 2564 generic.go:356] "Generic (PLEG): container finished" podID="d78f4cb7-4651-41f2-9e69-d109ebfdd3cc" containerID="b1fc71e0134b3e7ad88797e54a48afd62a77e4b0d6dfceddfa60e7f8256e1d88" exitCode=0 Jul 13 21:58:21.742625 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:21.742266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" event={"ID":"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc","Type":"ContainerDied","Data":"b1fc71e0134b3e7ad88797e54a48afd62a77e4b0d6dfceddfa60e7f8256e1d88"} Jul 13 21:58:22.885835 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:22.885808 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:22.976415 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:22.976370 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca\") pod \"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc\" (UID: \"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc\") " Jul 13 21:58:22.976747 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:22.976721 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d78f4cb7-4651-41f2-9e69-d109ebfdd3cc" (UID: "d78f4cb7-4651-41f2-9e69-d109ebfdd3cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:58:23.077853 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:23.077821 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d78f4cb7-4651-41f2-9e69-d109ebfdd3cc-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:58:23.756657 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:23.756626 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" Jul 13 21:58:23.756657 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:23.756641 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04c954-65zv9" event={"ID":"d78f4cb7-4651-41f2-9e69-d109ebfdd3cc","Type":"ContainerDied","Data":"36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9"} Jul 13 21:58:23.756967 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:23.756686 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36e2e8216b811a3a081e83cc88793db9a28c8bf426288486e1c81f56dc9fafd9" Jul 13 21:58:30.038818 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:30.038775 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:34.692606 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:34.692569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:40.527182 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:40.527113 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:40.626964 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:40.626917 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:45.375681 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:45.375645 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:45.597754 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:45.597717 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:45.911133 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:45.911095 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:46.210685 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:46.210558 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:46.338616 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:46.338540 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:49.475483 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:49.475443 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:49.572702 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:49.572666 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:49.750399 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:49.750308 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:49.936458 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:49.936408 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:50.423518 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:50.423472 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:50.519606 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:50.519569 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:53.991379 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:53.991345 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:54.376997 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:54.376959 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:54.734597 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:54.734512 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:54.936952 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:54.936913 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:56.815649 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:56.815592 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:57.624034 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:57.623967 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:57.714960 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:57.714922 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:57.927770 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:57.927679 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:58.247446 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:58.247322 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:58.469348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:58.469315 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:58:58.600792 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:58:58.600732 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:02.127700 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:02.127651 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:04.489872 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:04.489825 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:04.595731 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:04.595688 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:10.000846 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.000799 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh"] Jul 13 21:59:10.001348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.001275 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78f4cb7-4651-41f2-9e69-d109ebfdd3cc" containerName="revoke-keys" Jul 13 21:59:10.001348 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.001288 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78f4cb7-4651-41f2-9e69-d109ebfdd3cc" containerName="revoke-keys" Jul 13 21:59:10.001475 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.001360 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78f4cb7-4651-41f2-9e69-d109ebfdd3cc" containerName="revoke-keys" Jul 13 21:59:10.004538 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.004515 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:10.006814 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.006795 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:59:10.011587 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.011559 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh"] Jul 13 21:59:10.127254 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.127209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh\" (UID: \"0e8985a1-9dce-47e5-9e8f-d83806dd8221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:10.228383 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.228350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh\" (UID: \"0e8985a1-9dce-47e5-9e8f-d83806dd8221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:10.229138 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.229094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh\" (UID: \"0e8985a1-9dce-47e5-9e8f-d83806dd8221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:10.317361 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.317077 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:10.523802 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.523768 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:10.621825 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.621721 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:10.669811 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:10.669781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh"] Jul 13 21:59:10.671500 ip-10-0-131-109 kubenswrapper[2564]: W0713 21:59:10.671446 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e8985a1_9dce_47e5_9e8f_d83806dd8221.slice/crio-8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb WatchSource:0}: Error finding container 8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb: Status 404 returned error can't find the container with id 8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb Jul 13 21:59:11.043398 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:11.043363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" event={"ID":"0e8985a1-9dce-47e5-9e8f-d83806dd8221","Type":"ContainerStarted","Data":"927c5e5478316312dc4d26cd8d17802646a08276e97e377994abfb434e290554"} Jul 13 21:59:11.043893 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:11.043421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" event={"ID":"0e8985a1-9dce-47e5-9e8f-d83806dd8221","Type":"ContainerStarted","Data":"8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb"} Jul 13 21:59:11.059862 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:11.059794 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" podStartSLOduration=2.05977339 podStartE2EDuration="2.05977339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:11.057230832 +0000 UTC m=+2227.111765419" watchObservedRunningTime="2026-07-13 21:59:11.05977339 +0000 UTC m=+2227.114307978" Jul 13 21:59:12.052748 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:12.052708 2564 generic.go:356] "Generic (PLEG): container finished" podID="0e8985a1-9dce-47e5-9e8f-d83806dd8221" containerID="927c5e5478316312dc4d26cd8d17802646a08276e97e377994abfb434e290554" exitCode=0 Jul 13 21:59:12.053192 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:12.052763 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" event={"ID":"0e8985a1-9dce-47e5-9e8f-d83806dd8221","Type":"ContainerDied","Data":"927c5e5478316312dc4d26cd8d17802646a08276e97e377994abfb434e290554"} Jul 13 21:59:13.196698 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:13.196674 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:13.358790 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:13.358706 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca\") pod \"0e8985a1-9dce-47e5-9e8f-d83806dd8221\" (UID: \"0e8985a1-9dce-47e5-9e8f-d83806dd8221\") " Jul 13 21:59:13.359100 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:13.359078 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e8985a1-9dce-47e5-9e8f-d83806dd8221" (UID: "0e8985a1-9dce-47e5-9e8f-d83806dd8221"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:59:13.459621 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:13.459587 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e8985a1-9dce-47e5-9e8f-d83806dd8221-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:59:14.065746 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:14.065716 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" Jul 13 21:59:14.065920 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:14.065752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh" event={"ID":"0e8985a1-9dce-47e5-9e8f-d83806dd8221","Type":"ContainerDied","Data":"8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb"} Jul 13 21:59:14.065920 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:14.065796 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c078a3ece2de6c55633155404422ed8b4d18a3b846674d92eba0edeef454efb" Jul 13 21:59:19.338186 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:19.338147 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:19.436918 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:19.436878 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:20.692773 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:20.692733 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:20.793050 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:20.792996 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:20.950980 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:20.950881 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:25.086297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:25.086259 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:30.965427 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:30.965341 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:31.064586 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:31.064551 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:35.737984 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:35.737933 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:35.954887 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:35.954853 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:36.057592 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.057558 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k"] Jul 13 21:59:36.058074 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.058056 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e8985a1-9dce-47e5-9e8f-d83806dd8221" containerName="revoke-keys" Jul 13 21:59:36.058194 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.058077 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e8985a1-9dce-47e5-9e8f-d83806dd8221" containerName="revoke-keys" Jul 13 21:59:36.058251 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.058215 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e8985a1-9dce-47e5-9e8f-d83806dd8221" containerName="revoke-keys" Jul 13 21:59:36.061495 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.061471 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:36.064170 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.064136 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 21:59:36.068998 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.068972 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k"] Jul 13 21:59:36.173701 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.173664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k\" (UID: \"e3c132dd-59aa-42ed-8412-ebe360205a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:36.274986 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.274949 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k\" (UID: \"e3c132dd-59aa-42ed-8412-ebe360205a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:36.275636 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.275617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k\" (UID: \"e3c132dd-59aa-42ed-8412-ebe360205a7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:36.374148 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.374039 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:36.541620 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.540783 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k"] Jul 13 21:59:36.776523 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.776485 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:36.868106 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:36.868072 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:37.206069 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:37.206033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" event={"ID":"e3c132dd-59aa-42ed-8412-ebe360205a7f","Type":"ContainerStarted","Data":"1c0e94aff5e5ce6b7e2cf449b6b37d25e6270d95e60fee94d2ad92b46801672c"} Jul 13 21:59:37.206297 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:37.206090 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" event={"ID":"e3c132dd-59aa-42ed-8412-ebe360205a7f","Type":"ContainerStarted","Data":"97d55c318ab685f8f8e92f17a90ed2357937c108780cae7bb05c56a23d918c2b"} Jul 13 21:59:37.222929 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:37.222871 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" podStartSLOduration=1.2228544430000001 podStartE2EDuration="1.222854443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:37.221221331 +0000 UTC m=+2253.275755921" watchObservedRunningTime="2026-07-13 21:59:37.222854443 +0000 UTC m=+2253.277389030" Jul 13 21:59:38.213180 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:38.213147 2564 generic.go:356] "Generic (PLEG): container finished" podID="e3c132dd-59aa-42ed-8412-ebe360205a7f" containerID="1c0e94aff5e5ce6b7e2cf449b6b37d25e6270d95e60fee94d2ad92b46801672c" exitCode=0 Jul 13 21:59:38.213594 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:38.213222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" event={"ID":"e3c132dd-59aa-42ed-8412-ebe360205a7f","Type":"ContainerDied","Data":"1c0e94aff5e5ce6b7e2cf449b6b37d25e6270d95e60fee94d2ad92b46801672c"} Jul 13 21:59:39.357883 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:39.357858 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:39.402641 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:39.402607 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca\") pod \"e3c132dd-59aa-42ed-8412-ebe360205a7f\" (UID: \"e3c132dd-59aa-42ed-8412-ebe360205a7f\") " Jul 13 21:59:39.402987 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:39.402966 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3c132dd-59aa-42ed-8412-ebe360205a7f" (UID: "e3c132dd-59aa-42ed-8412-ebe360205a7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 21:59:39.503709 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:39.503611 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c132dd-59aa-42ed-8412-ebe360205a7f-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 21:59:40.226679 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:40.226640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" event={"ID":"e3c132dd-59aa-42ed-8412-ebe360205a7f","Type":"ContainerDied","Data":"97d55c318ab685f8f8e92f17a90ed2357937c108780cae7bb05c56a23d918c2b"} Jul 13 21:59:40.226679 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:40.226673 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k" Jul 13 21:59:40.226923 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:40.226702 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97d55c318ab685f8f8e92f17a90ed2357937c108780cae7bb05c56a23d918c2b" Jul 13 21:59:40.387349 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:40.387318 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:40.450442 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:40.450414 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:46.545094 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:46.545046 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:51.146880 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:51.146828 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:56.946292 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:56.946252 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 21:59:57.043091 ip-10-0-131-109 kubenswrapper[2564]: I0713 21:59:57.043056 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:00.140083 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.140042 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n"] Jul 13 22:00:00.140487 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.140476 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c132dd-59aa-42ed-8412-ebe360205a7f" containerName="revoke-keys" Jul 13 22:00:00.140487 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.140487 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c132dd-59aa-42ed-8412-ebe360205a7f" containerName="revoke-keys" Jul 13 22:00:00.140577 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.140564 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c132dd-59aa-42ed-8412-ebe360205a7f" containerName="revoke-keys" Jul 13 22:00:00.143789 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.143771 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" Jul 13 22:00:00.146481 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.146455 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 22:00:00.152162 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.152135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n"] Jul 13 22:00:00.155843 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.155822 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" Jul 13 22:00:00.303021 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.302990 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n"] Jul 13 22:00:00.304667 ip-10-0-131-109 kubenswrapper[2564]: W0713 22:00:00.304634 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73132749_ac3b_456d_bd4a_3e9b5c75d467.slice/crio-907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552 WatchSource:0}: Error finding container 907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552: Status 404 returned error can't find the container with id 907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552 Jul 13 22:00:00.346768 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:00.346735 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" event={"ID":"73132749-ac3b-456d-bd4a-3e9b5c75d467","Type":"ContainerStarted","Data":"907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552"} Jul 13 22:00:01.353610 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:01.353575 2564 generic.go:356] "Generic (PLEG): container finished" podID="73132749-ac3b-456d-bd4a-3e9b5c75d467" containerID="1e98bfa90c9cb296f3b7c124877587a817a343b3513744ce357e684ece2575cf" exitCode=0 Jul 13 22:00:01.353978 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:01.353659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" event={"ID":"73132749-ac3b-456d-bd4a-3e9b5c75d467","Type":"ContainerDied","Data":"1e98bfa90c9cb296f3b7c124877587a817a343b3513744ce357e684ece2575cf"} Jul 13 22:00:01.839446 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:01.839412 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:02.051354 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:02.051320 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:02.503270 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:02.503244 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" Jul 13 22:00:02.733529 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:02.733427 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:02.832862 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:02.832823 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:03.004702 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:03.004613 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:03.366533 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:03.366497 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" Jul 13 22:00:03.366533 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:03.366505 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733000-kx66n" event={"ID":"73132749-ac3b-456d-bd4a-3e9b5c75d467","Type":"ContainerDied","Data":"907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552"} Jul 13 22:00:03.366755 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:03.366565 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="907a48cdc03e58df25f0dfe0484209152b9be72736ecbd919145249abddaa552" Jul 13 22:00:05.926374 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:05.926337 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:06.018973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:06.018930 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:06.117651 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:06.117616 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:06.444696 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:06.444657 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:06.696838 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:06.696735 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:06.804090 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:06.804058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:10.415182 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:10.415144 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:10.725646 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:10.725609 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:11.051924 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:11.051885 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:11.409055 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:11.408966 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:13.966466 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:13.966427 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.066805 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:14.066757 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.196481 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:14.196448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.674050 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:14.674013 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.771103 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:14.771066 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:14.871582 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:14.871522 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:18.444608 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:18.444570 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.255264 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:28.255228 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:28.356902 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:28.356842 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:33.195445 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:33.195405 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:35.622544 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:35.622493 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:00:35.724641 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:00:35.724598 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.423129 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:01:14.423058 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:14.480246 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:01:14.480212 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:23.849183 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:01:23.849148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:58.357607 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:01:58.357573 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:01:58.460184 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:01:58.460148 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:20.066912 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:03:20.066872 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:21.198519 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:03:21.198478 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:23.803958 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:03:23.803914 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:25.412080 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:03:25.412043 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:03:51.515397 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:03:51.515345 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:05.310053 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:05.309958 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:05.411497 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:05.411462 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:27.956419 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:27.956389 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:28.794926 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:28.794889 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:34.815852 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:34.815814 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:52.355714 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:52.355676 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:04:54.539349 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:04:54.539281 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:15.054365 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:15.054328 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:15.156018 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:15.155966 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:28.971808 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:28.971732 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:29.072627 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:29.072593 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:36.550097 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:36.550051 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:36.650483 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:36.650448 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:40.594538 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:40.594504 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:05:41.396558 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:05:41.396502 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:00.142469 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.142432 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx"] Jul 13 22:06:00.143915 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.143885 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73132749-ac3b-456d-bd4a-3e9b5c75d467" containerName="cleanup" Jul 13 22:06:00.144081 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.144070 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="73132749-ac3b-456d-bd4a-3e9b5c75d467" containerName="cleanup" Jul 13 22:06:00.144425 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.144410 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="73132749-ac3b-456d-bd4a-3e9b5c75d467" containerName="cleanup" Jul 13 22:06:00.148711 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.148686 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:00.151894 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.151863 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 22:06:00.153968 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.153945 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx"] Jul 13 22:06:00.226758 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.226720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx\" (UID: \"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:00.327286 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.327254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx\" (UID: \"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:00.327948 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.327928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx\" (UID: \"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:00.460453 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.460363 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:00.605567 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.605538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx"] Jul 13 22:06:00.606599 ip-10-0-131-109 kubenswrapper[2564]: W0713 22:06:00.606568 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1a7a8e_f6e6_4fde_99bc_7710d0c15b9a.slice/crio-58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa WatchSource:0}: Error finding container 58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa: Status 404 returned error can't find the container with id 58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa Jul 13 22:06:00.914665 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:00.914632 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:01.017295 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:01.017264 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:01.530342 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:01.530306 2564 generic.go:356] "Generic (PLEG): container finished" podID="1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a" containerID="a6b0030e3bb2acde05d9412fe7b453018ca388da3d9fcdc19325b50c15899915" exitCode=0 Jul 13 22:06:01.530780 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:01.530391 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" event={"ID":"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a","Type":"ContainerDied","Data":"a6b0030e3bb2acde05d9412fe7b453018ca388da3d9fcdc19325b50c15899915"} Jul 13 22:06:01.530780 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:01.530444 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" event={"ID":"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a","Type":"ContainerStarted","Data":"58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa"} Jul 13 22:06:02.675721 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:02.675695 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:02.752019 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:02.751986 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca\") pod \"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a\" (UID: \"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a\") " Jul 13 22:06:02.752411 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:02.752391 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a" (UID: "1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:06:02.853613 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:02.853529 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 22:06:03.543351 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:03.543324 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" Jul 13 22:06:03.543351 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:03.543338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx" event={"ID":"1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a","Type":"ContainerDied","Data":"58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa"} Jul 13 22:06:03.543565 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:03.543380 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58714ea0f8ae3d0a826bb60363b6acef06d89665f468fa235bb13d2ad65094fa" Jul 13 22:06:10.570647 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:10.570600 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:15.217719 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:15.217684 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:21.267036 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:21.266999 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:21.367339 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:21.367302 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:25.906577 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:25.906546 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:26.114692 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.114650 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:26.238439 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.238341 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z"] Jul 13 22:06:26.238781 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.238765 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a" containerName="revoke-keys" Jul 13 22:06:26.238781 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.238780 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a" containerName="revoke-keys" Jul 13 22:06:26.238912 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.238847 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a" containerName="revoke-keys" Jul 13 22:06:26.242049 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.242026 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:26.245138 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.245100 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vjqr5\"" Jul 13 22:06:26.248567 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.248544 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z"] Jul 13 22:06:26.379349 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.379310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z\" (UID: \"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:26.480600 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.480558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z\" (UID: \"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:26.481284 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.481264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z\" (UID: \"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:26.555011 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.554974 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:26.699269 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.699236 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z"] Jul 13 22:06:26.700871 ip-10-0-131-109 kubenswrapper[2564]: W0713 22:06:26.700844 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cb9b9e_d8f8_433e_ab08_c82bbce57b05.slice/crio-cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4 WatchSource:0}: Error finding container cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4: Status 404 returned error can't find the container with id cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4 Jul 13 22:06:26.882888 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.882617 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:26.983707 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:26.983670 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:27.691531 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:27.691502 2564 generic.go:356] "Generic (PLEG): container finished" podID="a5cb9b9e-d8f8-433e-ab08-c82bbce57b05" containerID="91a12c821d2543ba6ce6670ce2c547dfa1cf46104414244aa3ac0294d2476882" exitCode=0 Jul 13 22:06:27.691689 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:27.691541 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" event={"ID":"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05","Type":"ContainerDied","Data":"91a12c821d2543ba6ce6670ce2c547dfa1cf46104414244aa3ac0294d2476882"} Jul 13 22:06:27.691689 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:27.691585 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" event={"ID":"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05","Type":"ContainerStarted","Data":"cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4"} Jul 13 22:06:28.844156 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:28.844113 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:29.005589 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.005493 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca\") pod \"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05\" (UID: \"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05\") " Jul 13 22:06:29.005819 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.005793 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5cb9b9e-d8f8-433e-ab08-c82bbce57b05" (UID: "a5cb9b9e-d8f8-433e-ab08-c82bbce57b05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 22:06:29.106677 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.106638 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cb9b9e-d8f8-433e-ab08-c82bbce57b05-maas-api-service-ca\") on node \"ip-10-0-131-109.ec2.internal\" DevicePath \"\"" Jul 13 22:06:29.705704 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.705662 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" event={"ID":"a5cb9b9e-d8f8-433e-ab08-c82bbce57b05","Type":"ContainerDied","Data":"cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4"} Jul 13 22:06:29.705704 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.705702 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z" Jul 13 22:06:29.705922 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:29.705726 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd2d7cb20f724c9759ff6a35090d5ab1d2847ec43422631b7ed7de5dc90291e4" Jul 13 22:06:37.162530 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:37.162481 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:41.440021 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:41.439984 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:48.996878 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:48.996846 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:49.125755 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:49.125720 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:52.129789 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:52.129755 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:52.352717 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:52.352684 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:06:58.149542 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:58.149455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77f465dbd8-ljrrr_632d0e2e-b098-4c98-9d7e-dec12b6439fd/manager/0.log" Jul 13 22:06:58.261880 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:58.261855 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-74smk_aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6/server/0.log" Jul 13 22:06:58.383164 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:58.383109 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-k7dmc_cfbe5ee8-0fac-4087-ba7f-0305ccffdc25/manager/0.log" Jul 13 22:06:58.733594 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:58.733570 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f98b9f4-z4rb6_ba7cdbc2-36bc-4ea8-81de-b7ede8d24548/copy-manifests/0.log" Jul 13 22:06:58.757986 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:58.757953 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f98b9f4-z4rb6_ba7cdbc2-36bc-4ea8-81de-b7ede8d24548/manager/0.log" Jul 13 22:06:59.113793 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:59.113762 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df54695bf-kkqbj_8c2231d1-4fbf-4ee0-a91b-8e63c493a8d8/maas-api/0.log" Jul 13 22:06:59.231758 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:59.231728 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732985-qb5vs_d924d256-abc1-4b83-9e3a-ba482cbe84eb/cleanup/0.log" Jul 13 22:06:59.337928 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:59.337900 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733000-kx66n_73132749-ac3b-456d-bd4a-3e9b5c75d467/cleanup/0.log" Jul 13 22:06:59.668619 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:59.668588 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-09f5ac-6drzh_0e8985a1-9dce-47e5-9e8f-d83806dd8221/revoke-keys/0.log" Jul 13 22:06:59.785652 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:06:59.785623 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-f5029e-j4j6k_e3c132dd-59aa-42ed-8412-ebe360205a7f/revoke-keys/0.log" Jul 13 22:07:00.007552 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.007464 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-6100d7-h48kj_9f13fe94-0d16-478d-b374-e1f4ea23ad93/revoke-keys/0.log" Jul 13 22:07:00.112709 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.112675 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-04c954-65zv9_d78f4cb7-4651-41f2-9e69-d109ebfdd3cc/revoke-keys/0.log" Jul 13 22:07:00.442733 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.442702 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-bfec26b9-27df4_35b4152f-eadc-4dbf-80d6-da60c8dde2e4/revoke-keys/0.log" Jul 13 22:07:00.554922 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.554890 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-44ef67-dpctx_1d1a7a8e-f6e6-4fde-99bc-7710d0c15b9a/revoke-keys/0.log" Jul 13 22:07:00.661608 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.661579 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-f0e328-nm89z_a5cb9b9e-d8f8-433e-ab08-c82bbce57b05/revoke-keys/0.log" Jul 13 22:07:00.769442 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:00.769414 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rkhg5_fd26bcf8-936c-4027-a4d6-8df6c55f33c6/postgres/0.log" Jul 13 22:07:01.518973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.518942 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/util/0.log" Jul 13 22:07:01.526655 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.526630 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/pull/0.log" Jul 13 22:07:01.534806 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.534782 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/extract/0.log" Jul 13 22:07:01.648022 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.647991 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/util/0.log" Jul 13 22:07:01.655099 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.655067 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/pull/0.log" Jul 13 22:07:01.661735 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.661710 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/extract/0.log" Jul 13 22:07:01.769926 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.769843 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/util/0.log" Jul 13 22:07:01.776401 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.776363 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/pull/0.log" Jul 13 22:07:01.782679 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.782658 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/extract/0.log" Jul 13 22:07:01.895031 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.895000 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/util/0.log" Jul 13 22:07:01.901875 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.901851 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/pull/0.log" Jul 13 22:07:01.908485 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:01.908464 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/extract/0.log" Jul 13 22:07:02.264561 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:02.264529 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zv9nm_f8d00ec0-1415-4a01-a499-1dc97089dfbd/manager/0.log" Jul 13 22:07:02.376341 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:02.376315 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7j9qv_a2f6cc87-1f8b-4545-ad84-ab9b0645b23f/kuadrant-console-plugin/0.log" Jul 13 22:07:02.495634 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:02.495606 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h8fd7_ade546ec-b72a-4b4b-b5b3-33a59ca32a49/registry-server/0.log" Jul 13 22:07:02.613838 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:02.613752 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7qnxs_471a5897-ee37-48a7-9517-f5f2a3d7f1d0/manager/0.log" Jul 13 22:07:02.843850 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:02.843817 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gxmds_c51bf87e-7af5-4b09-8a18-997e2a5d4c78/manager/0.log" Jul 13 22:07:03.295706 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:03.295676 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mc94w_00e7168b-cb97-4168-8b82-72cc999d3272/discovery/0.log" Jul 13 22:07:03.518498 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:03.518461 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6964b4f475-lgvc2_13951b00-017d-482f-b3a9-6f0bc0c4b09d/kube-auth-proxy/0.log" Jul 13 22:07:03.748288 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:03.748192 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-cff55_eef1f64f-57af-4854-a4a6-c2405a59d24a/payload-pre-processing/0.log" Jul 13 22:07:04.309904 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:04.309874 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-766dc_6a23f191-6c8f-4b18-b6bb-bce9e038dc42/main/0.log" Jul 13 22:07:04.537128 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:04.537087 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wtpfn_59694b11-e8c3-48cf-880a-025d2bee8836/main/0.log" Jul 13 22:07:04.649098 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:04.649008 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblsq55_845b5989-2469-4248-959b-1de00f8b52e6/main/0.log" Jul 13 22:07:08.836014 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:08.835978 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:08.904213 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:08.904171 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" limit=1024 Jul 13 22:07:11.838517 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:11.838477 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ffpgk_5209694d-9769-4c3f-be50-c900e8628640/global-pull-secret-syncer/0.log" Jul 13 22:07:11.947761 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:11.947725 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-72bf7_c1e3bd8a-2beb-44a1-88af-c230ed453992/konnectivity-agent/0.log" Jul 13 22:07:12.044871 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:12.044841 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-109.ec2.internal_3bea4fb0630134d8ede642fba0ebbd5f/haproxy/0.log" Jul 13 22:07:16.606787 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.606761 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/extract/0.log" Jul 13 22:07:16.633063 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.633029 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/util/0.log" Jul 13 22:07:16.655152 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.655111 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nbqmc_7c876446-9a3d-4188-9d32-b0553e0352ac/pull/0.log" Jul 13 22:07:16.686863 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.686837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/extract/0.log" Jul 13 22:07:16.711662 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.711631 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/util/0.log" Jul 13 22:07:16.735013 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.734983 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jvt7q_7b5454d9-64eb-4263-9fd4-38c83560fc85/pull/0.log" Jul 13 22:07:16.778632 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.778607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/extract/0.log" Jul 13 22:07:16.807611 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.807586 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/util/0.log" Jul 13 22:07:16.834042 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.834013 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ckb4t_8efbf1d6-b12b-47df-9ba7-acb8be596f5e/pull/0.log" Jul 13 22:07:16.873382 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.873294 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/extract/0.log" Jul 13 22:07:16.899016 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.898985 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/util/0.log" Jul 13 22:07:16.919373 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:16.919338 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15xj69_9f759ab1-627c-4bb2-bc0a-eb7706ea547a/pull/0.log" Jul 13 22:07:17.574174 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:17.574145 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zv9nm_f8d00ec0-1415-4a01-a499-1dc97089dfbd/manager/0.log" Jul 13 22:07:17.606107 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:17.606081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-7j9qv_a2f6cc87-1f8b-4545-ad84-ab9b0645b23f/kuadrant-console-plugin/0.log" Jul 13 22:07:17.666957 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:17.666932 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h8fd7_ade546ec-b72a-4b4b-b5b3-33a59ca32a49/registry-server/0.log" Jul 13 22:07:17.823870 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:17.823837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7qnxs_471a5897-ee37-48a7-9517-f5f2a3d7f1d0/manager/0.log" Jul 13 22:07:18.015464 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:18.015431 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-gxmds_c51bf87e-7af5-4b09-8a18-997e2a5d4c78/manager/0.log" Jul 13 22:07:19.801921 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.801891 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/alertmanager/0.log" Jul 13 22:07:19.848342 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.848254 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/config-reloader/0.log" Jul 13 22:07:19.886692 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.886660 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/kube-rbac-proxy-web/0.log" Jul 13 22:07:19.918300 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.918270 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/kube-rbac-proxy/0.log" Jul 13 22:07:19.950913 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.950886 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/kube-rbac-proxy-metric/0.log" Jul 13 22:07:19.988402 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:19.988376 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/prom-label-proxy/0.log" Jul 13 22:07:20.021870 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.021841 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee0e473f-5618-45de-9a8a-8662cf1b831e/init-config-reloader/0.log" Jul 13 22:07:20.193161 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.193028 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d8d59bc8b-znns2_69fbd780-2856-42d8-acb3-2fa4b8edffb9/metrics-server/0.log" Jul 13 22:07:20.236311 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.236277 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-lvtd6_0ece67f8-49ce-48d1-996d-8484567109e1/monitoring-plugin/0.log" Jul 13 22:07:20.273355 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.273329 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89gnt_2f98f5c5-bf03-4e64-865b-fd1db772aee8/node-exporter/0.log" Jul 13 22:07:20.298264 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.298239 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89gnt_2f98f5c5-bf03-4e64-865b-fd1db772aee8/kube-rbac-proxy/0.log" Jul 13 22:07:20.324376 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.324352 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89gnt_2f98f5c5-bf03-4e64-865b-fd1db772aee8/init-textfile/0.log" Jul 13 22:07:20.572628 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.572594 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-85zgl_f7e4ee8d-c771-445e-93fb-d46ab7b7d234/kube-rbac-proxy-main/0.log" Jul 13 22:07:20.615751 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.615726 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-85zgl_f7e4ee8d-c771-445e-93fb-d46ab7b7d234/kube-rbac-proxy-self/0.log" Jul 13 22:07:20.643553 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.643513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-85zgl_f7e4ee8d-c771-445e-93fb-d46ab7b7d234/openshift-state-metrics/0.log" Jul 13 22:07:20.887502 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.887408 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jp4kz_771a6d2e-c05e-4686-b555-9ec29f446787/prometheus-operator/0.log" Jul 13 22:07:20.905575 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.905551 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jp4kz_771a6d2e-c05e-4686-b555-9ec29f446787/kube-rbac-proxy/0.log" Jul 13 22:07:20.957954 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.957920 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cd9488bbb-nqzst_be01b040-2b51-4664-a2ac-823f1a060d9d/telemeter-client/0.log" Jul 13 22:07:20.979575 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:20.979551 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cd9488bbb-nqzst_be01b040-2b51-4664-a2ac-823f1a060d9d/reload/0.log" Jul 13 22:07:21.003827 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.003798 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cd9488bbb-nqzst_be01b040-2b51-4664-a2ac-823f1a060d9d/kube-rbac-proxy/0.log" Jul 13 22:07:21.844317 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.844289 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm"] Jul 13 22:07:21.844710 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.844699 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cb9b9e-d8f8-433e-ab08-c82bbce57b05" containerName="revoke-keys" Jul 13 22:07:21.844756 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.844712 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cb9b9e-d8f8-433e-ab08-c82bbce57b05" containerName="revoke-keys" Jul 13 22:07:21.844792 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.844778 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cb9b9e-d8f8-433e-ab08-c82bbce57b05" containerName="revoke-keys" Jul 13 22:07:21.847913 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.847896 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.850413 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.850377 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jl7zj\"/\"default-dockercfg-qtxmd\"" Jul 13 22:07:21.851318 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.851296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jl7zj\"/\"kube-root-ca.crt\"" Jul 13 22:07:21.851318 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.851318 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jl7zj\"/\"openshift-service-ca.crt\"" Jul 13 22:07:21.858222 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.858198 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm"] Jul 13 22:07:21.884040 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.884012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwwf\" (UniqueName: \"kubernetes.io/projected/a022f29f-d7b2-4b04-abc4-2feaaab425b0-kube-api-access-stwwf\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.884222 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.884080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-podres\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.884222 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.884175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-sys\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.884222 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.884209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-lib-modules\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.884337 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.884307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-proc\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985564 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985522 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-podres\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985564 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985570 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-sys\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-lib-modules\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-proc\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985654 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwwf\" (UniqueName: \"kubernetes.io/projected/a022f29f-d7b2-4b04-abc4-2feaaab425b0-kube-api-access-stwwf\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-sys\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985694 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-podres\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985754 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-proc\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.985973 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.985771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a022f29f-d7b2-4b04-abc4-2feaaab425b0-lib-modules\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:21.994442 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:21.994410 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwwf\" (UniqueName: \"kubernetes.io/projected/a022f29f-d7b2-4b04-abc4-2feaaab425b0-kube-api-access-stwwf\") pod \"perf-node-gather-daemonset-6r4bm\" (UID: \"a022f29f-d7b2-4b04-abc4-2feaaab425b0\") " pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:22.159838 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:22.159760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:22.299369 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:22.299339 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm"] Jul 13 22:07:22.300339 ip-10-0-131-109 kubenswrapper[2564]: W0713 22:07:22.300313 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda022f29f_d7b2_4b04_abc4_2feaaab425b0.slice/crio-8d28d52c02e8c020cbf551f0fa7daac58e9ae0753ae360005386c1c1b8a6ad0a WatchSource:0}: Error finding container 8d28d52c02e8c020cbf551f0fa7daac58e9ae0753ae360005386c1c1b8a6ad0a: Status 404 returned error can't find the container with id 8d28d52c02e8c020cbf551f0fa7daac58e9ae0753ae360005386c1c1b8a6ad0a Jul 13 22:07:23.019062 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:23.019024 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" event={"ID":"a022f29f-d7b2-4b04-abc4-2feaaab425b0","Type":"ContainerStarted","Data":"f884d9e29bf74cc4889d5afaf38de855e3e07133fa7ea17f0f721607a241196e"} Jul 13 22:07:23.019550 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:23.019081 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" event={"ID":"a022f29f-d7b2-4b04-abc4-2feaaab425b0","Type":"ContainerStarted","Data":"8d28d52c02e8c020cbf551f0fa7daac58e9ae0753ae360005386c1c1b8a6ad0a"} Jul 13 22:07:23.019550 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:23.019102 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:23.039676 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:23.039620 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" podStartSLOduration=2.039603582 podStartE2EDuration="2.039603582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:07:23.03644877 +0000 UTC m=+2719.090983356" watchObservedRunningTime="2026-07-13 22:07:23.039603582 +0000 UTC m=+2719.094138170" Jul 13 22:07:23.434631 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:23.434557 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d76b96744-cpfx9_7098a4db-4ac7-4835-bf90-afee7399ac46/console/0.log" Jul 13 22:07:25.067039 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:25.067007 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5m5z_08597cb4-981d-487d-a5a4-893aa674ea05/dns/0.log" Jul 13 22:07:25.088617 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:25.088594 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5m5z_08597cb4-981d-487d-a5a4-893aa674ea05/kube-rbac-proxy/0.log" Jul 13 22:07:25.258678 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:25.258643 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wrdfd_77492200-648f-408d-8463-d57da27d2b17/dns-node-resolver/0.log" Jul 13 22:07:25.857460 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:25.857431 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nt5rd_46100359-9719-40a0-a391-72bae71f86ab/node-ca/0.log" Jul 13 22:07:27.034085 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:27.034049 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mc94w_00e7168b-cb97-4168-8b82-72cc999d3272/discovery/0.log" Jul 13 22:07:27.092631 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:27.092599 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6964b4f475-lgvc2_13951b00-017d-482f-b3a9-6f0bc0c4b09d/kube-auth-proxy/0.log" Jul 13 22:07:27.195645 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:27.195613 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-cff55_eef1f64f-57af-4854-a4a6-c2405a59d24a/payload-pre-processing/0.log" Jul 13 22:07:27.994010 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:27.993977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-98wz5_74c6ecbc-f0ec-4696-8965-8e142fc71f51/serve-healthcheck-canary/0.log" Jul 13 22:07:28.564283 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:28.564254 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2hv_ee17311c-3732-451b-aa58-6dc841415566/kube-rbac-proxy/0.log" Jul 13 22:07:28.586285 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:28.586261 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2hv_ee17311c-3732-451b-aa58-6dc841415566/exporter/0.log" Jul 13 22:07:28.609364 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:28.609339 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2hv_ee17311c-3732-451b-aa58-6dc841415566/extractor/0.log" Jul 13 22:07:29.035031 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:29.035004 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jl7zj/perf-node-gather-daemonset-6r4bm" Jul 13 22:07:31.403270 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:31.403232 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77f465dbd8-ljrrr_632d0e2e-b098-4c98-9d7e-dec12b6439fd/manager/0.log" Jul 13 22:07:31.466701 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:31.466672 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-74smk_aedaad2f-86a4-4d1d-8b36-ca7b24fe2cf6/server/0.log" Jul 13 22:07:31.583830 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:31.583796 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-k7dmc_cfbe5ee8-0fac-4087-ba7f-0305ccffdc25/manager/0.log" Jul 13 22:07:31.824914 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:31.824881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f98b9f4-z4rb6_ba7cdbc2-36bc-4ea8-81de-b7ede8d24548/manager/0.log" Jul 13 22:07:31.868000 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:31.867968 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f98b9f4-z4rb6_ba7cdbc2-36bc-4ea8-81de-b7ede8d24548/copy-manifests/0.log" Jul 13 22:07:33.285743 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:33.285645 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mc94w_00e7168b-cb97-4168-8b82-72cc999d3272/discovery/0.log" Jul 13 22:07:33.319849 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:33.319820 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6964b4f475-lgvc2_13951b00-017d-482f-b3a9-6f0bc0c4b09d/kube-auth-proxy/0.log" Jul 13 22:07:33.414350 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:33.414320 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-cff55_eef1f64f-57af-4854-a4a6-c2405a59d24a/payload-pre-processing/0.log" Jul 13 22:07:34.908013 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:34.907982 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mc94w_00e7168b-cb97-4168-8b82-72cc999d3272/discovery/0.log" Jul 13 22:07:34.986606 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:34.986581 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6964b4f475-lgvc2_13951b00-017d-482f-b3a9-6f0bc0c4b09d/kube-auth-proxy/0.log" Jul 13 22:07:35.079296 ip-10-0-131-109 kubenswrapper[2564]: I0713 22:07:35.079260 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-cff55_eef1f64f-57af-4854-a4a6-c2405a59d24a/payload-pre-processing/0.log"