Jul 09 07:02:25.197706 ip-10-0-143-42 systemd[1]: Starting Kubernetes Kubelet... Jul 09 07:02:25.617461 ip-10-0-143-42 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 07:02:25.617461 ip-10-0-143-42 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 07:02:25.617461 ip-10-0-143-42 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 07:02:25.617461 ip-10-0-143-42 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 07:02:25.617461 ip-10-0-143-42 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 07:02:25.618746 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.618630 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 07:02:25.625083 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625064 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.625083 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625083 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625087 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625091 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625095 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625099 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625104 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625107 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625110 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625117 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625121 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625124 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625127 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625129 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625132 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625135 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625138 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625141 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625144 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625147 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625150 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.625155 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625152 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625155 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625158 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625161 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625166 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625170 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625174 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625177 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625180 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625183 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625186 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625191 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625195 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625198 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625201 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625204 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625208 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625211 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625214 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.625684 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625217 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625220 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625224 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625228 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625230 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625233 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625236 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625238 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625242 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625244 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625247 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625250 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625253 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625256 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625258 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625263 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625266 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625269 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625271 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.626183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625274 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625277 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625280 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625283 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625286 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625288 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625291 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625294 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625297 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625300 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625302 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625305 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625308 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625312 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625315 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625318 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625321 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625324 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625327 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625330 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.626729 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625333 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625335 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625338 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625341 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625344 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625347 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625350 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625353 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625356 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625359 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625361 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625366 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625370 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625373 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625376 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625379 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625381 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625385 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625387 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.625390 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.627294 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626187 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626194 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626197 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626200 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626205 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626209 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626212 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626215 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626218 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626221 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626224 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626227 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626230 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626233 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626236 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626239 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626242 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626244 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626247 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.627818 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626250 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626253 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626256 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626259 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626262 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626265 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626267 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626270 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626273 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626276 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626278 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626281 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626286 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626289 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626292 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626296 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626299 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626302 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626306 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.628321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626310 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626313 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626316 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626319 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626322 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626325 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626328 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626331 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626334 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626337 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626340 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626342 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626345 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626348 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626351 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626354 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626358 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626362 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626365 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626368 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.628834 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626370 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626373 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626376 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626379 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626382 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626384 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626387 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626390 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626393 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626395 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626399 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626404 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626407 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626409 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626412 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626415 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626417 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626420 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626423 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626426 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.629372 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626429 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626432 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626434 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626439 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626442 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626445 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626451 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626454 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626458 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626461 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626478 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626482 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626485 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626489 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626492 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626495 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626498 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626500 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626503 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.629910 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626506 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626509 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626933 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626938 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626942 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626946 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626949 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626952 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626955 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626958 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626961 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626964 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626967 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626970 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626973 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626976 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626978 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626981 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626984 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626989 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.630397 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626992 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626995 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.626998 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627001 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627003 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627006 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627009 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627012 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627015 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627018 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627020 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627023 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627027 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627030 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627033 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627035 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627039 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627041 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627044 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627047 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.630960 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627050 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627052 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627055 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627059 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627062 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627065 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627068 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627071 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627073 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627078 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627082 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627087 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627090 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627093 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627097 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627101 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627104 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627107 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627109 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.631485 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627112 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627115 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627117 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627120 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627123 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627126 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627129 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627132 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627135 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627138 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627141 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627144 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627147 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627150 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627152 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627155 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627158 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627161 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627164 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627167 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.631982 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627170 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627172 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627175 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627178 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627182 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627184 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627187 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627190 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627194 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627199 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627203 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627206 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627209 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627212 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627215 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627218 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627221 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627224 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627227 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627229 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.632536 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627232 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.627235 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627313 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627321 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627327 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627334 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627340 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627346 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627352 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627360 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627365 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627369 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627372 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627376 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627379 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627382 2573 flags.go:64] FLAG: --cgroup-root="" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627386 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627390 2573 flags.go:64] FLAG: --client-ca-file="" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627393 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627396 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627400 2573 flags.go:64] FLAG: --cluster-domain="" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627405 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627408 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627412 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 07:02:25.633056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627416 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627420 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627423 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627426 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627430 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627433 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627436 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627439 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627442 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627445 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627450 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627453 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627456 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627459 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627462 2573 flags.go:64] FLAG: --enable-server="true" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627480 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627485 2573 flags.go:64] FLAG: --event-burst="100" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627489 2573 flags.go:64] FLAG: --event-qps="50" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627492 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627496 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627499 2573 flags.go:64] FLAG: --eviction-hard="" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627503 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627506 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627510 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627514 2573 flags.go:64] FLAG: --eviction-soft="" Jul 09 07:02:25.633698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627517 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627520 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627523 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627527 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627535 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627538 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627541 2573 flags.go:64] FLAG: --feature-gates="" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627545 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627549 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627552 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627556 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627559 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627562 2573 flags.go:64] FLAG: --help="false" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627566 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-143-42.ec2.internal" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627569 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627572 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627575 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627579 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627583 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627586 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627589 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627592 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627595 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 07:02:25.634372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627598 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627602 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627605 2573 flags.go:64] FLAG: --kube-reserved="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627608 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627612 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627615 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627618 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627621 2573 flags.go:64] FLAG: --lock-file="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627625 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627628 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627632 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627637 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627640 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627645 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627648 2573 flags.go:64] FLAG: --logging-format="text" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627651 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627654 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627657 2573 flags.go:64] FLAG: --manifest-url="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627660 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627665 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627668 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627672 2573 flags.go:64] FLAG: --max-pods="110" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627676 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627679 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627682 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 07:02:25.635289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627685 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627689 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627692 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627695 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627702 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627706 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627709 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627712 2573 flags.go:64] FLAG: --pod-cidr="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627715 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627722 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627725 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627728 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627732 2573 flags.go:64] FLAG: --port="10250" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627735 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627739 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-053e706cf1685ee7a" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627742 2573 flags.go:64] FLAG: --qos-reserved="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627745 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627749 2573 flags.go:64] FLAG: --register-node="true" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627752 2573 flags.go:64] FLAG: --register-with-taints="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627756 2573 flags.go:64] FLAG: --registry-burst="10" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627760 2573 flags.go:64] FLAG: --registry-qps="5" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627764 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627767 2573 flags.go:64] FLAG: --reserved-memory="" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627771 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627774 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 07:02:25.636386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627778 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627781 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627784 2573 flags.go:64] FLAG: --runonce="false" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627787 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627790 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627793 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627796 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627799 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627803 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627806 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627810 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627813 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627816 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627819 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627822 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627826 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627829 2573 flags.go:64] FLAG: --system-cgroups="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627833 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627839 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627842 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627845 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627850 2573 flags.go:64] FLAG: --tls-min-version="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627854 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627858 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627861 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 07:02:25.637276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627864 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627867 2573 flags.go:64] FLAG: --v="2" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627872 2573 flags.go:64] FLAG: --version="false" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627877 2573 flags.go:64] FLAG: --vmodule="" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627881 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.627885 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628010 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628014 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628017 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628021 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628024 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628027 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628030 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628033 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628036 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628039 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628041 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628044 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628047 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628051 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.637951 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628055 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628059 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628062 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628065 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628067 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628070 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628073 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628076 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628079 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628082 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628087 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628089 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628092 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628095 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628100 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628103 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628105 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628108 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628111 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628114 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.638643 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628116 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628119 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628122 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628125 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628127 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628130 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628133 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628136 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628139 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628142 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628144 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628148 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628150 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628153 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628156 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628159 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628161 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628164 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628167 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.639209 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628170 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628174 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628179 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628183 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628186 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628189 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628192 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628197 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628200 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628203 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628207 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628210 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628212 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628216 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628219 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628223 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628225 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628228 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628231 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.639716 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628234 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628237 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628239 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628242 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628245 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628248 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628250 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628253 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628256 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628259 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628262 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628264 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628267 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628270 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628272 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628275 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628280 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628284 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628288 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628291 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.640214 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628295 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628298 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628301 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628304 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628307 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628309 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.628312 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.629116 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.638637 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.638655 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.638670 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.638677 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638740 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638747 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.640745 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638751 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638755 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638758 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638761 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638764 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638767 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638770 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638772 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638775 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638778 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638781 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638784 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638787 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638790 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638792 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638796 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638799 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638802 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638806 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638809 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.641134 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638812 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638815 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638818 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638821 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638824 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638827 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638830 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638833 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638836 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638839 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638842 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638845 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638848 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638851 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638853 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638856 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638859 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638862 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638864 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638867 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.641679 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638870 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638873 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638876 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638878 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638881 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638884 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638887 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638890 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638893 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638895 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638899 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638902 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638905 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638908 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638911 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638914 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638917 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638920 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638923 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638926 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.642257 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638929 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638931 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638934 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638937 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638940 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638942 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638945 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638948 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638951 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638954 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638956 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638959 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638962 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638965 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638968 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638970 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638973 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638977 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638980 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638983 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.642816 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638986 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638989 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638991 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.638996 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639000 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639003 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639006 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639009 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639012 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639016 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639019 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639022 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639026 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639030 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639033 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639036 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.643321 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639038 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.639044 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639170 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639176 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639180 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639183 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639186 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639189 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639193 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639196 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639199 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639202 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639205 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639208 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639212 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 07:02:25.643865 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639215 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639218 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639222 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639225 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639228 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639231 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639234 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639238 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639242 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639246 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639249 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639252 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639255 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639257 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639261 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639264 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639267 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639270 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639273 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 07:02:25.644270 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639276 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639279 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639282 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639285 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639288 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639291 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639293 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639296 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639299 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639301 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639304 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639307 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639310 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639313 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639316 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639319 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639322 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639324 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639327 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 07:02:25.644785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639331 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639335 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639338 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639341 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639344 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639347 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639350 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639352 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639355 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639358 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639361 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639364 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639367 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639370 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639373 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639376 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639379 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639381 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639384 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639387 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 07:02:25.645279 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639389 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639392 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639395 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639397 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639400 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639403 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639406 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639411 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639414 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639417 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639419 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639422 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639425 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639428 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639430 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639433 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639436 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639439 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639442 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639445 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 07:02:25.645962 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639448 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639451 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639454 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639456 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639459 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639462 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639482 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:25.639486 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.639492 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.639608 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.641877 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.643509 2573 server.go:1019] "Starting client certificate rotation" Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.643642 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 07:02:25.646921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.643698 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 07:02:25.647303 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.646253 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 07:02:25.647518 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.647498 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 07:02:25.660875 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.660853 2573 log.go:25] "Validated CRI v1 runtime API" Jul 09 07:02:25.663212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.663195 2573 log.go:25] "Validated CRI v1 image API" Jul 09 07:02:25.665126 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.665111 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 07:02:25.669220 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.669196 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a14cc15f-3eed-460e-9ff4-ee08876109bb:/dev/nvme0n1p4 eef4acf7-377e-49c6-9ef0-fb0b752128ee:/dev/nvme0n1p3] Jul 09 07:02:25.669268 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.669220 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 07:02:25.676117 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.676093 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 07:02:25.677987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.677870 2573 manager.go:216] Machine: {Timestamp:2026-07-09 07:02:25.676409649 +0000 UTC m=+0.350266521 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098817 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26950ee28259ad4cad9002950c5d76 SystemUUID:ec26950e-e282-59ad-4cad-9002950c5d76 BootID:c6138cba-bcd9-498c-a0fc-5f1ea01c03a1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:d2:37:39:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:d2:37:39:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:6e:df:4a:fa:a7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 07:02:25.677987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.677984 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 07:02:25.678113 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.678073 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 07:02:25.681838 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.681808 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 07:02:25.681990 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.681842 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 07:02:25.682547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.682537 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 07:02:25.682580 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.682550 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 07:02:25.682580 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.682558 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 07:02:25.683590 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.683568 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 07:02:25.684972 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.684961 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 07:02:25.688031 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688019 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 09 07:02:25.688167 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688157 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 07:02:25.688200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688190 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 09 07:02:25.688229 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688206 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 07:02:25.688229 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688222 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 07:02:25.688283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688231 2573 kubelet.go:398] "Adding apiserver pod source" Jul 09 07:02:25.688283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688241 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 07:02:25.688722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688708 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 07:02:25.688774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.688737 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 07:02:25.691070 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.691051 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 07:02:25.692838 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.692822 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 07:02:25.692911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.692856 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 07:02:25.694233 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694220 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694237 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694244 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694250 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694260 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694266 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 07:02:25.694275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694276 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 07:02:25.694455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694282 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 07:02:25.694455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694289 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 07:02:25.694455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694296 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 07:02:25.694455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694306 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 07:02:25.694455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.694318 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 07:02:25.695175 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.695164 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 07:02:25.695175 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.695176 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 07:02:25.698039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.698016 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mxlk" Jul 09 07:02:25.698625 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.698612 2573 server.go:1268] "Started kubelet" Jul 09 07:02:25.698839 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.698778 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 07:02:25.698889 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.698866 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 07:02:25.698974 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.698708 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 07:02:25.699681 ip-10-0-143-42 systemd[1]: Started Kubernetes Kubelet. Jul 09 07:02:25.699842 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.699809 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 07:02:25.699951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.699934 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 07:02:25.700260 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.700167 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 07:02:25.700464 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.700437 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 07:02:25.700964 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.700949 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 09 07:02:25.705033 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.704094 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-42.ec2.internal.18c08d0e76a94d91 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-42.ec2.internal,UID:ip-10-0-143-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-42.ec2.internal,},FirstTimestamp:2026-07-09 07:02:25.698581905 +0000 UTC m=+0.372438745,LastTimestamp:2026-07-09 07:02:25.698581905 +0000 UTC m=+0.372438745,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-42.ec2.internal,}" Jul 09 07:02:25.705932 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.705916 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 07:02:25.706018 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.705935 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 07:02:25.706094 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.706075 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:25.706732 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.706713 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 07:02:25.707203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.706970 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 07:02:25.707203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.706991 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 07:02:25.707203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.706970 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 07:02:25.707203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.707163 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 09 07:02:25.707203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.707173 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 09 07:02:25.708903 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.708689 2573 factory.go:55] Registering systemd factory Jul 09 07:02:25.708903 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.708709 2573 factory.go:223] Registration of the systemd container factory successfully Jul 09 07:02:25.709106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709082 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mxlk" Jul 09 07:02:25.709181 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709088 2573 factory.go:153] Registering CRI-O factory Jul 09 07:02:25.709181 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709158 2573 factory.go:223] Registration of the crio container factory successfully Jul 09 07:02:25.709273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709253 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 07:02:25.709312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709304 2573 factory.go:103] Registering Raw factory Jul 09 07:02:25.709346 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.709321 2573 manager.go:1196] Started watching for new ooms in manager Jul 09 07:02:25.710170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.710146 2573 manager.go:349] Starting recovery of all containers Jul 09 07:02:25.715214 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.715175 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 07:02:25.715323 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.715253 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 07:02:25.722116 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.722092 2573 manager.go:354] Recovery completed Jul 09 07:02:25.726887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.726874 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:25.729147 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729129 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:25.729236 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729162 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:25.729236 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729182 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:25.729867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729849 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 07:02:25.729867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729864 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 07:02:25.729998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729878 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:25.729998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.729880 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 09 07:02:25.730666 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.730647 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:25.730726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.730676 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:25.730726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.730721 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:25.731437 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.731423 2573 policy_none.go:49] "None policy: Start" Jul 09 07:02:25.731525 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.731444 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 07:02:25.731525 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.731459 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 07:02:25.732753 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.732739 2573 policy_none.go:47] "Start" Jul 09 07:02:25.774231 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774054 2573 manager.go:337] "Starting Device Plugin manager" Jul 09 07:02:25.774344 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.774262 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 07:02:25.774344 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774278 2573 server.go:87] "Starting device plugin registration server" Jul 09 07:02:25.774688 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774672 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 07:02:25.774767 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774690 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 07:02:25.774816 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774788 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 07:02:25.774940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774910 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 07:02:25.774940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.774923 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 07:02:25.775903 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.775882 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 07:02:25.775993 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.775930 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:25.878502 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.878454 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:25.879703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.879567 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:25.879703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.879600 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:25.879703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.879613 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:25.879703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.879643 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-42.ec2.internal" Jul 09 07:02:25.888988 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.888970 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-42.ec2.internal" Jul 09 07:02:25.889058 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.888993 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-42.ec2.internal\": node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:25.902700 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.902679 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:25.909615 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.909588 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 07:02:25.911004 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.910980 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 07:02:25.911069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.911025 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 07:02:25.911069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.911046 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 07:02:25.911142 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:25.911078 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 07:02:25.913720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:25.913699 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 07:02:26.004062 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.004009 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.012108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.012037 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal"] Jul 09 07:02:26.012206 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.012118 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:26.013256 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.013238 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:26.013331 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.013273 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:26.013331 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.013283 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:26.014738 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.014725 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:26.014862 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.014847 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.014906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.014890 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:26.015608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015589 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:26.015608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015599 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:26.015720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015618 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:26.015720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015626 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:26.015720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015642 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:26.015720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.015628 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:26.017005 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.016990 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.017058 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.017017 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 07:02:26.017765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.017749 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 09 07:02:26.017861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.017775 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 07:02:26.017861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.017784 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 09 07:02:26.041446 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.041421 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-42.ec2.internal\" not found" node="ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.046204 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.046187 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-42.ec2.internal\" not found" node="ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.104978 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.104950 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.109672 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.109654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.109733 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.109681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.109733 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.109698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afa10e102bff134ec3a47ae4ffe0aeb1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"afa10e102bff134ec3a47ae4ffe0aeb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.205956 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.205924 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.210574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210557 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.210641 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.210695 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.210753 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afa10e102bff134ec3a47ae4ffe0aeb1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"afa10e102bff134ec3a47ae4ffe0aeb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.210808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42cd6bf80d78304b851b50404a315f6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"42cd6bf80d78304b851b50404a315f6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.210808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.210776 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/afa10e102bff134ec3a47ae4ffe0aeb1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"afa10e102bff134ec3a47ae4ffe0aeb1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.313726 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.313640 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.345776 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.345732 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.350102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.349791 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:26.414716 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.414683 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.515643 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.515598 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.615957 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.615866 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.643318 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.643292 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 07:02:26.643760 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.643476 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 07:02:26.716481 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.716455 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.719869 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.719843 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 06:57:26 +0000 UTC" deadline="2028-04-15 15:14:51.77874444 +0000 UTC" Jul 09 07:02:26.719946 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.719870 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15512h12m25.058878828s" Jul 09 07:02:26.774030 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.774002 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 07:02:26.816610 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.816573 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.917282 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:26.917251 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:26.987164 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:26.987124 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cd6bf80d78304b851b50404a315f6e.slice/crio-f169e463133fb7ba3e22bf1d00f5bdf188fd78174edfb8fc22c73e77eb5bc20a WatchSource:0}: Error finding container f169e463133fb7ba3e22bf1d00f5bdf188fd78174edfb8fc22c73e77eb5bc20a: Status 404 returned error can't find the container with id f169e463133fb7ba3e22bf1d00f5bdf188fd78174edfb8fc22c73e77eb5bc20a Jul 09 07:02:26.994153 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:26.994135 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:02:27.017694 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.017661 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:27.118334 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.118299 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 09 07:02:27.155324 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.155248 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 07:02:27.193937 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:27.193912 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa10e102bff134ec3a47ae4ffe0aeb1.slice/crio-1f7cd275c18582e027a3d8d5c577bdbf2bfb68857110e6a7f283bb6c4a8b6f56 WatchSource:0}: Error finding container 1f7cd275c18582e027a3d8d5c577bdbf2bfb68857110e6a7f283bb6c4a8b6f56: Status 404 returned error can't find the container with id 1f7cd275c18582e027a3d8d5c577bdbf2bfb68857110e6a7f283bb6c4a8b6f56 Jul 09 07:02:27.195516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.195384 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 07:02:27.206392 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.206372 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 09 07:02:27.215610 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.215593 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 07:02:27.216827 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.216813 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 09 07:02:27.227336 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.227320 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 07:02:27.691666 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.691626 2573 apiserver.go:52] "Watching apiserver" Jul 09 07:02:27.699799 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.699766 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 07:02:27.700313 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.700285 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-bvzsl","openshift-multus/multus-bkq24","openshift-network-operator/iptables-alerter-wfzcn","kube-system/global-pull-secret-syncer-4w92n","kube-system/konnectivity-agent-f7mth","kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m","openshift-cluster-node-tuning-operator/tuned-8mlws","openshift-multus/network-metrics-daemon-gvzjc","openshift-network-diagnostics/network-check-target-vf8f8","openshift-ovn-kubernetes/ovnkube-node-lzjwl","openshift-dns/node-resolver-ztdwp","openshift-image-registry/node-ca-dnh77","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal"] Jul 09 07:02:27.702574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.702549 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.705817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.704752 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 07:02:27.705817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.704896 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.705817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.705105 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.705817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.705119 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.705817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.705330 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rxb24\"" Jul 09 07:02:27.707270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.707249 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.707618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.707599 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qr77n\"" Jul 09 07:02:27.707949 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.707931 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.708376 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.707939 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 07:02:27.708734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.708715 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.708912 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.708892 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 07:02:27.710140 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.710119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.710242 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.710214 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:27.710845 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.710824 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.711024 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.711003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d4ns8\"" Jul 09 07:02:27.711085 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.711036 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.711085 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.711006 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 07:02:27.711286 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.711268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.713239 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.713219 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 07:02:27.713402 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.713333 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nlmgs\"" Jul 09 07:02:27.713502 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.713483 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 07:02:27.713571 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.713556 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.716180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.716064 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g7ckm\"" Jul 09 07:02:27.716373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.716359 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.716481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.716396 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.717895 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.717876 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:27.718006 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.717976 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:27.720423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.720394 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.721939 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.721911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-iptables-alerter-script\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.722035 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.721957 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-sys-fs\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.721991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b07514b-c2af-4d17-99a6-37e4102ac780-host\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.722319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722108 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.722319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722270 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-dbus\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.722319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.722521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722390 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-socket-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-device-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722455 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-registration-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-etc-selinux\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722552 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-host-slash\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljs58\" (UniqueName: \"kubernetes.io/projected/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-kube-api-access-ljs58\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8l76\" (UniqueName: \"kubernetes.io/projected/31456e72-04c6-45e7-a467-140000776779-kube-api-access-q8l76\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b07514b-c2af-4d17-99a6-37e4102ac780-serviceca\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722676 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twh8w\" (UniqueName: \"kubernetes.io/projected/0b07514b-c2af-4d17-99a6-37e4102ac780-kube-api-access-twh8w\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.722725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-kubelet-config\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.723065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722949 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 07:02:27.723065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722966 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.723065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722973 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 07:02:27.723247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.722981 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 07:02:27.723247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.723111 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.723247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.723118 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 07:02:27.723368 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.723344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5nfxj\"" Jul 09 07:02:27.724630 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.724169 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 07:02:27.725080 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.724908 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mzsms\"" Jul 09 07:02:27.725383 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.725367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:27.725489 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.725455 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:27.727225 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.727202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.728847 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.728824 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.729133 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.729108 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.729413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.729393 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cwxht\"" Jul 09 07:02:27.729583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.729565 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 07:02:27.729709 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.729693 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.730624 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.730607 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 07:02:27.730721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.730662 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 07:02:27.730782 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.730664 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 07:02:27.731099 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.731079 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 07:02:27.731099 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.731098 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-grvzv\"" Jul 09 07:02:27.731355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.731338 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 07:02:27.743697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.743672 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lssb8" Jul 09 07:02:27.751118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.751095 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 07:02:27.752023 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.751991 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lssb8" Jul 09 07:02:27.808248 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.808212 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 07:02:27.822946 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.822913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-registration-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.822958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.822988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8f5\" (UniqueName: \"kubernetes.io/projected/5d94dae0-522d-4c36-ad14-0b64b68efec7-kube-api-access-mv8f5\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-slash\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-registration-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823039 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-systemd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.823108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-bin\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b07514b-c2af-4d17-99a6-37e4102ac780-serviceca\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823164 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twh8w\" (UniqueName: \"kubernetes.io/projected/0b07514b-c2af-4d17-99a6-37e4102ac780-kube-api-access-twh8w\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823195 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823254 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-iptables-alerter-script\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-socket-dir-parent\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-netd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823348 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysconfig\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823371 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-systemd\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.823486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823398 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823512 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-socket-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.823556 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.823637 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:28.323607244 +0000 UTC m=+2.997464071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-etc-selinux\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-socket-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823744 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b07514b-c2af-4d17-99a6-37e4102ac780-serviceca\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-host-slash\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-system-cni-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-sys\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823890 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-host-slash\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823875 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-etc-selinux\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823937 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-host\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.823985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-k8s-cni-cncf-io\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824003 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-iptables-alerter-script\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824026 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-script-lib\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.824042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d977d649-07ee-4d4b-b64e-3f89c121db76-hosts-file\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-system-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8l76\" (UniqueName: \"kubernetes.io/projected/31456e72-04c6-45e7-a467-140000776779-kube-api-access-q8l76\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824155 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-cni-binary-copy\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovn-node-metrics-cert\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d977d649-07ee-4d4b-b64e-3f89c121db76-tmp-dir\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824252 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-run\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824345 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-tmp\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-kubelet-config\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824398 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-etc-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824458 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-kubelet-config\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-config\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-sys-fs\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-device-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-cnibin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.824832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-sys-fs\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-hostroot\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824727 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-conf-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824765 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-var-lib-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-device-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ffe5743-2bfd-4475-b676-351e1d6fefbe-agent-certs\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824829 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9w6\" (UniqueName: \"kubernetes.io/projected/acf17b4e-bde3-42fc-9019-eaa60ea69b84-kube-api-access-xn9w6\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45m7l\" (UniqueName: \"kubernetes.io/projected/74999f2c-f351-4a55-ba48-337671bc61df-kube-api-access-45m7l\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824921 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-netns\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824956 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-etc-kubernetes\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.824981 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69bg6\" (UniqueName: \"kubernetes.io/projected/d977d649-07ee-4d4b-b64e-3f89c121db76-kube-api-access-69bg6\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825005 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-conf\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825030 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-lib-modules\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825067 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-var-lib-kubelet\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825117 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.825734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-systemd-units\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-ovn\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe5743-2bfd-4475-b676-351e1d6fefbe-konnectivity-ca\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825244 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-modprobe-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mw9d\" (UniqueName: \"kubernetes.io/projected/d6141152-a2f7-48a5-99f1-0dd084e92b7f-kube-api-access-8mw9d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b07514b-c2af-4d17-99a6-37e4102ac780-host\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-dbus\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825349 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-os-release\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-multus\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825399 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-netns\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825463 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljs58\" (UniqueName: \"kubernetes.io/projected/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-kube-api-access-ljs58\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825515 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19428e47-7c22-4917-a531-e27501b88251-dbus\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825527 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b07514b-c2af-4d17-99a6-37e4102ac780-host\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-os-release\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-kubelet\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-multus-certs\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.826633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825647 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-log-socket\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825686 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-env-overrides\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-cnibin\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-bin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-daemon-config\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-kubelet\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825837 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-node-log\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mrfs\" (UniqueName: \"kubernetes.io/projected/4ef49bae-e1df-44b4-a2f0-aa9050189388-kube-api-access-2mrfs\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-kubernetes\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.825996 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-tuned\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.826033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.826091 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.827504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.826096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/31456e72-04c6-45e7-a467-140000776779-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.833303 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.833274 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 07:02:27.837861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.837794 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljs58\" (UniqueName: \"kubernetes.io/projected/13baef2a-e34e-4c82-a97f-a9a1fdc72ed9-kube-api-access-ljs58\") pod \"iptables-alerter-wfzcn\" (UID: \"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9\") " pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:27.837861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.837829 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8l76\" (UniqueName: \"kubernetes.io/projected/31456e72-04c6-45e7-a467-140000776779-kube-api-access-q8l76\") pod \"aws-ebs-csi-driver-node-bk75m\" (UID: \"31456e72-04c6-45e7-a467-140000776779\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:27.837861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.837794 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twh8w\" (UniqueName: \"kubernetes.io/projected/0b07514b-c2af-4d17-99a6-37e4102ac780-kube-api-access-twh8w\") pod \"node-ca-dnh77\" (UID: \"0b07514b-c2af-4d17-99a6-37e4102ac780\") " pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:27.916747 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.916682 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" event={"ID":"afa10e102bff134ec3a47ae4ffe0aeb1","Type":"ContainerStarted","Data":"1f7cd275c18582e027a3d8d5c577bdbf2bfb68857110e6a7f283bb6c4a8b6f56"} Jul 09 07:02:27.918307 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.918277 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"42cd6bf80d78304b851b50404a315f6e","Type":"ContainerStarted","Data":"f169e463133fb7ba3e22bf1d00f5bdf188fd78174edfb8fc22c73e77eb5bc20a"} Jul 09 07:02:27.926673 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926647 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-cnibin\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-bin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-daemon-config\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-kubelet\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926759 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-cnibin\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926770 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-bin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.926802 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-kubelet\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926844 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-node-log\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mrfs\" (UniqueName: \"kubernetes.io/projected/4ef49bae-e1df-44b4-a2f0-aa9050189388-kube-api-access-2mrfs\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-node-log\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926922 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-kubernetes\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-tuned\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.926990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927026 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927041 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8f5\" (UniqueName: \"kubernetes.io/projected/5d94dae0-522d-4c36-ad14-0b64b68efec7-kube-api-access-mv8f5\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-slash\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-systemd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927113 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-bin\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927143 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927172 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.927197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927201 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-socket-dir-parent\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927228 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-netd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-daemon-config\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysconfig\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-systemd\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927325 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-systemd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927350 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-system-cni-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927358 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-sys\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-host\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-socket-dir-parent\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927435 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-slash\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-k8s-cni-cncf-io\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927489 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-k8s-cni-cncf-io\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927493 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-netd\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-script-lib\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d977d649-07ee-4d4b-b64e-3f89c121db76-hosts-file\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927557 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-cni-bin\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.928271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-system-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-cni-binary-copy\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927607 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-system-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovn-node-metrics-cert\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d977d649-07ee-4d4b-b64e-3f89c121db76-tmp-dir\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927663 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysconfig\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927742 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-systemd\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-run\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-tmp\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-run\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927813 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-etc-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927864 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-config\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-cnibin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927918 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-hostroot\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-conf-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.929276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.927956 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928002 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-var-lib-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ffe5743-2bfd-4475-b676-351e1d6fefbe-agent-certs\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9w6\" (UniqueName: \"kubernetes.io/projected/acf17b4e-bde3-42fc-9019-eaa60ea69b84-kube-api-access-xn9w6\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928123 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45m7l\" (UniqueName: \"kubernetes.io/projected/74999f2c-f351-4a55-ba48-337671bc61df-kube-api-access-45m7l\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-netns\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928257 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-netns\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928282 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928305 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-system-cni-dir\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-kubernetes\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-script-lib\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-etc-kubernetes\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928497 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-sys\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69bg6\" (UniqueName: \"kubernetes.io/projected/d977d649-07ee-4d4b-b64e-3f89c121db76-kube-api-access-69bg6\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-conf\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928559 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d977d649-07ee-4d4b-b64e-3f89c121db76-tmp-dir\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.930190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928583 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-lib-modules\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-var-lib-kubelet\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928651 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928678 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-systemd-units\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-ovn\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928746 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe5743-2bfd-4475-b676-351e1d6fefbe-konnectivity-ca\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-modprobe-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mw9d\" (UniqueName: \"kubernetes.io/projected/d6141152-a2f7-48a5-99f1-0dd084e92b7f-kube-api-access-8mw9d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928839 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-etc-kubernetes\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-os-release\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928922 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-conf-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928957 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74999f2c-f351-4a55-ba48-337671bc61df-os-release\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928975 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovnkube-config\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d977d649-07ee-4d4b-b64e-3f89c121db76-hosts-file\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929173 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-host\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929253 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-multus-cni-dir\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929276 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929344 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-etc-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.931563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929375 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-conf\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-multus\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929444 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-netns\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929503 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-sysctl-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-os-release\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929547 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-kubelet\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929560 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-systemd-units\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-multus-certs\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929542 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-modprobe-d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-log-socket\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-env-overrides\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929858 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d94dae0-522d-4c36-ad14-0b64b68efec7-cni-binary-copy\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.929950 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-var-lib-kubelet\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.929977 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930099 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6141152-a2f7-48a5-99f1-0dd084e92b7f-lib-modules\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.930127 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:28.430108222 +0000 UTC m=+3.103965077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:27.932560 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930133 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ef49bae-e1df-44b4-a2f0-aa9050189388-env-overrides\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930144 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-run-ovn\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930158 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/74999f2c-f351-4a55-ba48-337671bc61df-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930193 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-os-release\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.928960 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-var-lib-openvswitch\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930219 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-run-multus-certs\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-cni-multus\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.930455 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-cnibin\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.931967 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-host-run-netns\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932050 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-hostroot\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932121 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ef49bae-e1df-44b4-a2f0-aa9050189388-ovn-node-metrics-cert\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932134 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ef49bae-e1df-44b4-a2f0-aa9050189388-log-socket\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932380 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ffe5743-2bfd-4475-b676-351e1d6fefbe-konnectivity-ca\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-etc-tuned\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932695 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6141152-a2f7-48a5-99f1-0dd084e92b7f-tmp\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.932687 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d94dae0-522d-4c36-ad14-0b64b68efec7-host-var-lib-kubelet\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.933166 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.933196 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.933212 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:27.933413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.933222 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ffe5743-2bfd-4475-b676-351e1d6fefbe-agent-certs\") pod \"konnectivity-agent-f7mth\" (UID: \"6ffe5743-2bfd-4475-b676-351e1d6fefbe\") " pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:27.934401 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:27.933279 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:28.433261445 +0000 UTC m=+3.107118287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:27.938787 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.936834 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8f5\" (UniqueName: \"kubernetes.io/projected/5d94dae0-522d-4c36-ad14-0b64b68efec7-kube-api-access-mv8f5\") pod \"multus-bkq24\" (UID: \"5d94dae0-522d-4c36-ad14-0b64b68efec7\") " pod="openshift-multus/multus-bkq24" Jul 09 07:02:27.939652 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.939556 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mrfs\" (UniqueName: \"kubernetes.io/projected/4ef49bae-e1df-44b4-a2f0-aa9050189388-kube-api-access-2mrfs\") pod \"ovnkube-node-lzjwl\" (UID: \"4ef49bae-e1df-44b4-a2f0-aa9050189388\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:27.940470 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.940412 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mw9d\" (UniqueName: \"kubernetes.io/projected/d6141152-a2f7-48a5-99f1-0dd084e92b7f-kube-api-access-8mw9d\") pod \"tuned-8mlws\" (UID: \"d6141152-a2f7-48a5-99f1-0dd084e92b7f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:27.941571 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.941507 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45m7l\" (UniqueName: \"kubernetes.io/projected/74999f2c-f351-4a55-ba48-337671bc61df-kube-api-access-45m7l\") pod \"multus-additional-cni-plugins-bvzsl\" (UID: \"74999f2c-f351-4a55-ba48-337671bc61df\") " pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:27.942119 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.941962 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69bg6\" (UniqueName: \"kubernetes.io/projected/d977d649-07ee-4d4b-b64e-3f89c121db76-kube-api-access-69bg6\") pod \"node-resolver-ztdwp\" (UID: \"d977d649-07ee-4d4b-b64e-3f89c121db76\") " pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:27.943258 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:27.943234 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9w6\" (UniqueName: \"kubernetes.io/projected/acf17b4e-bde3-42fc-9019-eaa60ea69b84-kube-api-access-xn9w6\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:28.020411 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.020378 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" Jul 09 07:02:28.028853 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.028505 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dnh77" Jul 09 07:02:28.036132 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.036101 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31456e72_04c6_45e7_a467_140000776779.slice/crio-00b16437efdec45ff500935c1e710f17340045490b6e3b12e6857f3162653127 WatchSource:0}: Error finding container 00b16437efdec45ff500935c1e710f17340045490b6e3b12e6857f3162653127: Status 404 returned error can't find the container with id 00b16437efdec45ff500935c1e710f17340045490b6e3b12e6857f3162653127 Jul 09 07:02:28.037768 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.037627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wfzcn" Jul 09 07:02:28.038746 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.038718 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b07514b_c2af_4d17_99a6_37e4102ac780.slice/crio-3f07c476b184516415d7cfca768993d11b55de67f3349a453b2d6816c611fc18 WatchSource:0}: Error finding container 3f07c476b184516415d7cfca768993d11b55de67f3349a453b2d6816c611fc18: Status 404 returned error can't find the container with id 3f07c476b184516415d7cfca768993d11b55de67f3349a453b2d6816c611fc18 Jul 09 07:02:28.045569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.045536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:28.048331 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.048291 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13baef2a_e34e_4c82_a97f_a9a1fdc72ed9.slice/crio-69b71098940905005cd6bb7966b47e1827fb7cdd2fd9eb72a513fa81c3f4c604 WatchSource:0}: Error finding container 69b71098940905005cd6bb7966b47e1827fb7cdd2fd9eb72a513fa81c3f4c604: Status 404 returned error can't find the container with id 69b71098940905005cd6bb7966b47e1827fb7cdd2fd9eb72a513fa81c3f4c604 Jul 09 07:02:28.053912 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.053885 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8mlws" Jul 09 07:02:28.055578 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.055549 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffe5743_2bfd_4475_b676_351e1d6fefbe.slice/crio-cfd9197f3499d38d01246701cb8baef01911d64a1ff31396d2eab9f8b1522733 WatchSource:0}: Error finding container cfd9197f3499d38d01246701cb8baef01911d64a1ff31396d2eab9f8b1522733: Status 404 returned error can't find the container with id cfd9197f3499d38d01246701cb8baef01911d64a1ff31396d2eab9f8b1522733 Jul 09 07:02:28.062584 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.062344 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" Jul 09 07:02:28.065153 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.065130 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6141152_a2f7_48a5_99f1_0dd084e92b7f.slice/crio-389e7f3d5edc2861e6f90d9ce1023841a060500a14d04304d5f774ff3b213130 WatchSource:0}: Error finding container 389e7f3d5edc2861e6f90d9ce1023841a060500a14d04304d5f774ff3b213130: Status 404 returned error can't find the container with id 389e7f3d5edc2861e6f90d9ce1023841a060500a14d04304d5f774ff3b213130 Jul 09 07:02:28.070395 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.070212 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bkq24" Jul 09 07:02:28.072778 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.072687 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74999f2c_f351_4a55_ba48_337671bc61df.slice/crio-c9d24a409362acda781265450ba413b14eb94dc2423094e575d6dceba4fe480a WatchSource:0}: Error finding container c9d24a409362acda781265450ba413b14eb94dc2423094e575d6dceba4fe480a: Status 404 returned error can't find the container with id c9d24a409362acda781265450ba413b14eb94dc2423094e575d6dceba4fe480a Jul 09 07:02:28.078289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.078266 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:28.080766 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.080733 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d94dae0_522d_4c36_ad14_0b64b68efec7.slice/crio-7c7989bf0b12e01b5c2a6b707ce74560715eda49e037cf7442aef45c4e7b1d96 WatchSource:0}: Error finding container 7c7989bf0b12e01b5c2a6b707ce74560715eda49e037cf7442aef45c4e7b1d96: Status 404 returned error can't find the container with id 7c7989bf0b12e01b5c2a6b707ce74560715eda49e037cf7442aef45c4e7b1d96 Jul 09 07:02:28.085686 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.085666 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztdwp" Jul 09 07:02:28.333089 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.332995 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:28.333264 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.333109 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:28.333264 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.333193 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:29.333172059 +0000 UTC m=+4.007028904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:28.433683 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.433642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:28.433864 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.433751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:28.433917 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.433865 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:28.433917 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.433902 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:28.433917 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.433917 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:28.434042 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.433873 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:28.434042 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.433984 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:29.433964493 +0000 UTC m=+4.107821341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:28.434042 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.434035 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:29.434016821 +0000 UTC m=+4.107873672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:28.754721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.754612 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 06:57:28 +0000 UTC" deadline="2028-04-15 18:53:19.689982344 +0000 UTC" Jul 09 07:02:28.754721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.754664 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15515h50m50.935323518s" Jul 09 07:02:28.854782 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:28.854742 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef49bae_e1df_44b4_a2f0_aa9050189388.slice/crio-13bbd85475a92ce85d33a3cd58444f150f93a365bfef4f644c2ac62a8501301c WatchSource:0}: Error finding container 13bbd85475a92ce85d33a3cd58444f150f93a365bfef4f644c2ac62a8501301c: Status 404 returned error can't find the container with id 13bbd85475a92ce85d33a3cd58444f150f93a365bfef4f644c2ac62a8501301c Jul 09 07:02:28.912022 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.911993 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:28.912190 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.912124 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:28.912266 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.912191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:28.912343 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:28.912317 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:28.922367 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.922336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"13bbd85475a92ce85d33a3cd58444f150f93a365bfef4f644c2ac62a8501301c"} Jul 09 07:02:28.924140 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.924103 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerStarted","Data":"c9d24a409362acda781265450ba413b14eb94dc2423094e575d6dceba4fe480a"} Jul 09 07:02:28.925547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.925506 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dnh77" event={"ID":"0b07514b-c2af-4d17-99a6-37e4102ac780","Type":"ContainerStarted","Data":"3f07c476b184516415d7cfca768993d11b55de67f3349a453b2d6816c611fc18"} Jul 09 07:02:28.927449 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.927404 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bkq24" event={"ID":"5d94dae0-522d-4c36-ad14-0b64b68efec7","Type":"ContainerStarted","Data":"7c7989bf0b12e01b5c2a6b707ce74560715eda49e037cf7442aef45c4e7b1d96"} Jul 09 07:02:28.928860 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.928831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8mlws" event={"ID":"d6141152-a2f7-48a5-99f1-0dd084e92b7f","Type":"ContainerStarted","Data":"389e7f3d5edc2861e6f90d9ce1023841a060500a14d04304d5f774ff3b213130"} Jul 09 07:02:28.930374 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.930348 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f7mth" event={"ID":"6ffe5743-2bfd-4475-b676-351e1d6fefbe","Type":"ContainerStarted","Data":"cfd9197f3499d38d01246701cb8baef01911d64a1ff31396d2eab9f8b1522733"} Jul 09 07:02:28.932069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.932032 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wfzcn" event={"ID":"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9","Type":"ContainerStarted","Data":"69b71098940905005cd6bb7966b47e1827fb7cdd2fd9eb72a513fa81c3f4c604"} Jul 09 07:02:28.934098 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:28.934069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" event={"ID":"31456e72-04c6-45e7-a467-140000776779","Type":"ContainerStarted","Data":"00b16437efdec45ff500935c1e710f17340045490b6e3b12e6857f3162653127"} Jul 09 07:02:29.342927 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.342881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:29.343143 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.343121 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:29.343216 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.343188 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:31.34316989 +0000 UTC m=+6.017026718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:29.444339 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.444297 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:29.444543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.444363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:29.444634 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444613 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:29.444722 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444643 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:29.444722 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444658 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:29.444722 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444720 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:31.444702153 +0000 UTC m=+6.118558983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:29.444890 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444792 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:29.444890 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.444827 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:31.444814627 +0000 UTC m=+6.118671460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:29.743192 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:02:29.743149 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd977d649_07ee_4d4b_b64e_3f89c121db76.slice/crio-e62343effb8c224d3405fe5e531723bff8afc5fd1b020cb94b2146c8e016dbdd WatchSource:0}: Error finding container e62343effb8c224d3405fe5e531723bff8afc5fd1b020cb94b2146c8e016dbdd: Status 404 returned error can't find the container with id e62343effb8c224d3405fe5e531723bff8afc5fd1b020cb94b2146c8e016dbdd Jul 09 07:02:29.755160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.755096 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 06:57:28 +0000 UTC" deadline="2028-01-21 15:21:39.675290885 +0000 UTC" Jul 09 07:02:29.755160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.755133 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13472h19m9.920164013s" Jul 09 07:02:29.915120 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.914278 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:29.915120 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:29.914743 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:29.939223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.939179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztdwp" event={"ID":"d977d649-07ee-4d4b-b64e-3f89c121db76","Type":"ContainerStarted","Data":"e62343effb8c224d3405fe5e531723bff8afc5fd1b020cb94b2146c8e016dbdd"} Jul 09 07:02:29.943468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.943434 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" event={"ID":"afa10e102bff134ec3a47ae4ffe0aeb1","Type":"ContainerStarted","Data":"cd48137f59d06ce632205deda26a8b061680896b13cc2ccb20bce699f4e60efc"} Jul 09 07:02:29.957710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:29.957525 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" podStartSLOduration=2.9575044310000003 podStartE2EDuration="2.957504431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:29.957203721 +0000 UTC m=+4.631060590" watchObservedRunningTime="2026-07-09 07:02:29.957504431 +0000 UTC m=+4.631361285" Jul 09 07:02:30.912252 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:30.912207 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:30.912252 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:30.912249 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:30.912884 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:30.912363 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:30.912884 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:30.912539 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:30.961247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:30.961206 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"42cd6bf80d78304b851b50404a315f6e","Type":"ContainerStarted","Data":"cd95d2395bc51978a3ea8d6f366710477df85d1f7ff3a87ba868dd521a2f0d24"} Jul 09 07:02:31.362081 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:31.361985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:31.362351 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.362176 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:31.362351 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.362250 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:35.362227813 +0000 UTC m=+10.036084654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:31.462959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:31.462887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:31.462959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:31.462949 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:31.463191 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463140 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:31.463191 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463159 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:31.463191 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463173 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:31.463376 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463234 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:35.463214735 +0000 UTC m=+10.137071584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:31.463722 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463693 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:31.463841 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.463749 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:35.463735564 +0000 UTC m=+10.137592391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:31.912270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:31.912234 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:31.912473 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:31.912386 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:32.912129 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:32.912094 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:32.912314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:32.912138 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:32.912314 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:32.912236 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:32.912711 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:32.912668 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:33.914508 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:33.913310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:33.914508 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:33.913469 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:33.970580 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:33.970540 2573 generic.go:356] "Generic (PLEG): container finished" podID="42cd6bf80d78304b851b50404a315f6e" containerID="cd95d2395bc51978a3ea8d6f366710477df85d1f7ff3a87ba868dd521a2f0d24" exitCode=0 Jul 09 07:02:33.970762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:33.970592 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"42cd6bf80d78304b851b50404a315f6e","Type":"ContainerDied","Data":"cd95d2395bc51978a3ea8d6f366710477df85d1f7ff3a87ba868dd521a2f0d24"} Jul 09 07:02:34.912468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:34.911520 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:34.912468 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:34.911714 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:34.912468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:34.912218 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:34.912468 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:34.912412 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:35.401517 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:35.400938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:35.401517 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.401082 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:35.401517 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.401153 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:43.401135165 +0000 UTC m=+18.074992011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:35.501899 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:35.501817 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:35.501899 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:35.501882 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:35.502145 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.501970 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:35.502145 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.502055 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:43.50203508 +0000 UTC m=+18.175891907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:35.502145 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.502052 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:35.502145 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.502087 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:35.502145 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.502102 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:35.502371 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.502154 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:43.502136029 +0000 UTC m=+18.175992855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:35.912561 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:35.912489 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:35.912736 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:35.912610 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:36.911506 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:36.911470 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:36.911506 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:36.911494 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:36.912033 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:36.911618 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:36.912033 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:36.911716 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:37.912509 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:37.911990 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:37.912509 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:37.912134 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:38.911789 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:38.911708 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:38.911924 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:38.911827 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:38.911924 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:38.911880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:38.912026 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:38.911968 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:39.911622 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:39.911589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:39.912046 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:39.911732 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:40.911410 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:40.911367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:40.911604 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:40.911501 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:40.911604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:40.911576 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:40.911969 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:40.911689 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:41.911643 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:41.911604 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:41.911817 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:41.911761 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:42.911887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:42.911850 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:42.912317 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:42.911888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:42.912317 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:42.911988 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:42.912317 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:42.912121 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:43.459205 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:43.459165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:43.459378 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.459295 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:43.459378 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.459356 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:59.459342871 +0000 UTC m=+34.133199697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:43.559860 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:43.559816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:43.560049 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:43.559902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:43.560049 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.559961 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:43.560049 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.559987 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:43.560049 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.560000 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:43.560049 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.560004 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:43.560300 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.560060 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:02:59.56003975 +0000 UTC m=+34.233896575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:43.560300 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.560080 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:02:59.560069671 +0000 UTC m=+34.233926495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:43.912037 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:43.912002 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:43.912493 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:43.912149 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:44.911726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:44.911694 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:44.911726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:44.911711 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:44.911957 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:44.911806 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:44.911957 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:44.911940 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:45.912627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:45.912588 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:45.913087 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:45.912711 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:46.911239 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:46.911213 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:46.911340 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:46.911320 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:46.911409 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:46.911351 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:46.911453 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:46.911427 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:47.911771 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:47.911563 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:47.912455 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:47.911890 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:47.998641 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:47.998610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"42cd6bf80d78304b851b50404a315f6e","Type":"ContainerStarted","Data":"368153ec203ee7e2ab985cb4f50360acfd4f1db85cc11fb6495e1963cba1b2be"} Jul 09 07:02:48.000683 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.000665 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:02:48.000971 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.000952 2573 generic.go:356] "Generic (PLEG): container finished" podID="4ef49bae-e1df-44b4-a2f0-aa9050189388" containerID="d53d5beaa499f681c67acbadd6afe623749ccc05644b5dc7aee3716a405854df" exitCode=1 Jul 09 07:02:48.001030 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.001015 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"879526a8df4830ced51c493367c31f7d6ea184a1106837b7bc3b9a6ec72e1458"} Jul 09 07:02:48.001105 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.001090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"bd8ffa761fa630adf37a6114abd86943a9e6832f7afd2313ac0d4984d4bcf492"} Jul 09 07:02:48.001163 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.001148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerDied","Data":"d53d5beaa499f681c67acbadd6afe623749ccc05644b5dc7aee3716a405854df"} Jul 09 07:02:48.001218 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.001204 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"51c2befe9b6eb5b0962caa85897de4b2547b1777a9084aa4fbe02d9b001d2bdb"} Jul 09 07:02:48.002581 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.002563 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="fd6405db85d54ac0691795b6513ebe6817d22b8586f2aa2283c5d8dc7523260f" exitCode=0 Jul 09 07:02:48.002674 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.002640 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"fd6405db85d54ac0691795b6513ebe6817d22b8586f2aa2283c5d8dc7523260f"} Jul 09 07:02:48.007272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.007233 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dnh77" event={"ID":"0b07514b-c2af-4d17-99a6-37e4102ac780","Type":"ContainerStarted","Data":"27ff45a2d2e5ebc3999ce173a0f3f99a8d467d9df2cd4a9748a3c804d3bf8218"} Jul 09 07:02:48.008833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.008807 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bkq24" event={"ID":"5d94dae0-522d-4c36-ad14-0b64b68efec7","Type":"ContainerStarted","Data":"329c35e7264db14fe1f0c1361017d105f83bb97ba6b006280a8418e91fae09bf"} Jul 09 07:02:48.010335 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.010308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8mlws" event={"ID":"d6141152-a2f7-48a5-99f1-0dd084e92b7f","Type":"ContainerStarted","Data":"5c97a047b184cee7be76adec6e40011cdee84f29bd136e51dfc5eebe467a1130"} Jul 09 07:02:48.011908 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.011878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f7mth" event={"ID":"6ffe5743-2bfd-4475-b676-351e1d6fefbe","Type":"ContainerStarted","Data":"a2989d842c573b667f7773cf4b9cf9057ed5d2282303f5f168883a9a2fe5f9e8"} Jul 09 07:02:48.013434 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.013382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" event={"ID":"31456e72-04c6-45e7-a467-140000776779","Type":"ContainerStarted","Data":"19de7081b82b604bd8806eae23d0deae08bfc7abdc689146b730b979ad88810a"} Jul 09 07:02:48.014638 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.014605 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztdwp" event={"ID":"d977d649-07ee-4d4b-b64e-3f89c121db76","Type":"ContainerStarted","Data":"3927b865d9123d6fd2eb9ddf5f5d55af322904b89c351627698cb11fc1094875"} Jul 09 07:02:48.015755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.015719 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" podStartSLOduration=21.015708983 podStartE2EDuration="21.015708983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.015381901 +0000 UTC m=+22.689238748" watchObservedRunningTime="2026-07-09 07:02:48.015708983 +0000 UTC m=+22.689565829" Jul 09 07:02:48.044790 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.044749 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dnh77" podStartSLOduration=2.807899398 podStartE2EDuration="22.044736158s" totalImagesPullingTime="19.23683676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.027878968 +0000 UTC m=+22.701735825" watchObservedRunningTime="2026-07-09 07:02:48.044736158 +0000 UTC m=+22.718593004" Jul 09 07:02:48.067681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.067636 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8mlws" podStartSLOduration=2.858319917 podStartE2EDuration="22.067621583s" totalImagesPullingTime="19.209301666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.044494774 +0000 UTC m=+22.718351622" watchObservedRunningTime="2026-07-09 07:02:48.067621583 +0000 UTC m=+22.741478430" Jul 09 07:02:48.079477 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.079433 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f7mth" podStartSLOduration=3.232371324 podStartE2EDuration="22.079420722s" totalImagesPullingTime="18.847049398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.079349821 +0000 UTC m=+22.753206668" watchObservedRunningTime="2026-07-09 07:02:48.079420722 +0000 UTC m=+22.753277569" Jul 09 07:02:48.094329 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.094281 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bkq24" podStartSLOduration=2.886395135 podStartE2EDuration="22.094262714s" totalImagesPullingTime="19.207867579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.093485093 +0000 UTC m=+22.767341967" watchObservedRunningTime="2026-07-09 07:02:48.094262714 +0000 UTC m=+22.768119561" Jul 09 07:02:48.106906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.106860 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ztdwp" podStartSLOduration=4.578469566 podStartE2EDuration="22.106844574s" totalImagesPullingTime="17.528375008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:48.106234494 +0000 UTC m=+22.780091340" watchObservedRunningTime="2026-07-09 07:02:48.106844574 +0000 UTC m=+22.780701420" Jul 09 07:02:48.911410 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.911373 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:48.911573 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:48.911502 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:48.911573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:48.911540 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:48.911683 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:48.911618 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:49.019333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.019303 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" event={"ID":"31456e72-04c6-45e7-a467-140000776779","Type":"ContainerStarted","Data":"0027d870d810f3ee0a514d7d9f29048ea10c91720bc1de3118c35d7a9ddecf34"} Jul 09 07:02:49.022832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.022805 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:02:49.023150 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.023129 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 07:02:49.023265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.023240 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"18e75bbc40c6f02ab25dd6a8546588ab1ea00409061ed6d5f62fd8febc81e14a"} Jul 09 07:02:49.023340 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.023325 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"6df83e081d12c5db4e05d8e86f8c66997f5787141f00090f0ffb4812a59756b9"} Jul 09 07:02:49.024808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.024743 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wfzcn" event={"ID":"13baef2a-e34e-4c82-a97f-a9a1fdc72ed9","Type":"ContainerStarted","Data":"2902fa3db9024d8c5a996a05e20d0baf473b6738f10fdffd00a11515496f0f56"} Jul 09 07:02:49.038541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.038503 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wfzcn" podStartSLOduration=4.184476654 podStartE2EDuration="23.038489685s" totalImagesPullingTime="18.854013031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:49.038066125 +0000 UTC m=+23.711923005" watchObservedRunningTime="2026-07-09 07:02:49.038489685 +0000 UTC m=+23.712346531" Jul 09 07:02:49.787602 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.787495 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T07:02:49.023146186Z","UUID":"23964dd0-39c1-4fdb-aa6f-a8267da3f3a3","Handler":null,"Name":"","Endpoint":""} Jul 09 07:02:49.789003 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.788973 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 07:02:49.789131 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.789020 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 07:02:49.911916 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:49.911876 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:49.912082 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:49.912032 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:50.912371 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:50.912144 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:50.912852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:50.912188 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:50.912852 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:50.912497 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:50.912852 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:50.912543 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:51.032998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:51.032972 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:02:51.033462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:51.033431 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"d454e4dbd1d73442f07094b027ee61b61932daf7dd82aaf0f1b7a473f649a86b"} Jul 09 07:02:51.035699 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:51.035667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" event={"ID":"31456e72-04c6-45e7-a467-140000776779","Type":"ContainerStarted","Data":"68bef31ad20f0d585a9f6b35bbfeb9c090a8628c06f697177e47fe4022daad98"} Jul 09 07:02:51.053065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:51.053016 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bk75m" podStartSLOduration=3.511558569 podStartE2EDuration="25.05299918s" totalImagesPullingTime="21.541440611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:51.051326116 +0000 UTC m=+25.725182964" watchObservedRunningTime="2026-07-09 07:02:51.05299918 +0000 UTC m=+25.726856025" Jul 09 07:02:51.911333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:51.911308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:51.911525 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:51.911437 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:52.016818 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:52.016782 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:52.017456 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:52.017436 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:52.911966 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:52.911926 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:52.912142 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:52.912066 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:52.912142 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:52.912122 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:52.912252 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:52.912227 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:53.911462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:53.911268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:53.912119 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:53.911570 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:54.045080 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.044999 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:02:54.045419 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.045364 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"305f319e91fed8ad6c78253f66cdbaaa335ed5be6fa8764a3336505350f69d30"} Jul 09 07:02:54.045701 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.045677 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:54.045792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.045740 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:54.045903 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.045871 2573 scope.go:117] "RemoveContainer" containerID="d53d5beaa499f681c67acbadd6afe623749ccc05644b5dc7aee3716a405854df" Jul 09 07:02:54.047359 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.047339 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="cb5cf791c7c2aac0996fc5d6540e2402e07547c65dec729843f2b84b4d18ccdd" exitCode=0 Jul 09 07:02:54.047481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.047379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"cb5cf791c7c2aac0996fc5d6540e2402e07547c65dec729843f2b84b4d18ccdd"} Jul 09 07:02:54.062145 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.062126 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:54.911951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.911918 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:54.912435 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:54.911933 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:54.912435 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:54.912025 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:54.912435 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:54.912085 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:55.053041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.053013 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:02:55.053376 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.053351 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" event={"ID":"4ef49bae-e1df-44b4-a2f0-aa9050189388","Type":"ContainerStarted","Data":"c9e1886465754722868f3e515f9cc3ec067a4810dd6be6f8636923190c0f4ba2"} Jul 09 07:02:55.053652 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.053629 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:55.068607 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.068585 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:02:55.078209 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.078165 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" podStartSLOduration=10.411366686 podStartE2EDuration="29.078153033s" totalImagesPullingTime="18.666786347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:02:55.076620496 +0000 UTC m=+29.750477351" watchObservedRunningTime="2026-07-09 07:02:55.078153033 +0000 UTC m=+29.752009880" Jul 09 07:02:55.441971 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.441698 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vf8f8"] Jul 09 07:02:55.442113 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.442020 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:55.442151 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:55.442121 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:55.442495 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.442473 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4w92n"] Jul 09 07:02:55.442596 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.442557 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:55.442656 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:55.442640 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:55.443316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.443283 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvzjc"] Jul 09 07:02:55.443437 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:55.443409 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:55.443530 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:55.443513 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:56.057604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.057569 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="8df82a523e359fa96439c04103aeffffa179a41e2bac9a675947e40486ea221f" exitCode=0 Jul 09 07:02:56.058000 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.057647 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"8df82a523e359fa96439c04103aeffffa179a41e2bac9a675947e40486ea221f"} Jul 09 07:02:56.911910 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.911880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:56.912059 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.911921 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:56.912059 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.911971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:56.912059 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:56.911981 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:56.912059 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:56.912048 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:56.912198 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:56.912096 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:56.953137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.953108 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:56.953306 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.953228 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 07:02:56.953746 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:56.953722 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f7mth" Jul 09 07:02:58.064885 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:58.064854 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="7a43deba9887ff3caa617f24ad3bb1befc2de837983267ef1be08a94d3ed0912" exitCode=0 Jul 09 07:02:58.065490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:58.064894 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"7a43deba9887ff3caa617f24ad3bb1befc2de837983267ef1be08a94d3ed0912"} Jul 09 07:02:58.911815 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:58.911778 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:58.912018 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:58.911778 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:58.912018 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:58.911909 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vf8f8" podUID="4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f" Jul 09 07:02:58.912018 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:58.911788 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:58.912191 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:58.911972 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4w92n" podUID="19428e47-7c22-4917-a531-e27501b88251" Jul 09 07:02:58.912191 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:58.912095 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvzjc" podUID="acf17b4e-bde3-42fc-9019-eaa60ea69b84" Jul 09 07:02:59.478164 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:59.478128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:02:59.478636 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.478302 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:59.478636 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.478380 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret podName:19428e47-7c22-4917-a531-e27501b88251 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:31.478358671 +0000 UTC m=+66.152215502 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret") pod "global-pull-secret-syncer-4w92n" (UID: "19428e47-7c22-4917-a531-e27501b88251") : object "kube-system"/"original-pull-secret" not registered Jul 09 07:02:59.579409 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:59.579359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:02:59.579594 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:02:59.579429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:02:59.579594 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579526 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:59.579594 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579568 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 07:02:59.579594 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579589 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 07:02:59.579594 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579598 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wd45r for pod openshift-network-diagnostics/network-check-target-vf8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:02:59.579830 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579620 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs podName:acf17b4e-bde3-42fc-9019-eaa60ea69b84 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:31.579598929 +0000 UTC m=+66.253455767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs") pod "network-metrics-daemon-gvzjc" (UID: "acf17b4e-bde3-42fc-9019-eaa60ea69b84") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 07:02:59.579830 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:02:59.579648 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r podName:4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:31.579630426 +0000 UTC m=+66.253488429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wd45r" (UniqueName: "kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r") pod "network-check-target-vf8f8" (UID: "4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 07:03:00.131813 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.131741 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeReady" Jul 09 07:03:00.131960 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.131913 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 07:03:00.163914 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.163882 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:03:00.166137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.165973 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.167838 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.167818 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 07:03:00.168141 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.168117 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 07:03:00.168584 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.168562 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z7kjk\"" Jul 09 07:03:00.168681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.168609 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 07:03:00.173455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.173427 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 07:03:00.176636 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.176615 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pv68j"] Jul 09 07:03:00.178800 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.178779 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g4vhc"] Jul 09 07:03:00.178967 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.178947 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.180756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.180734 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:03:00.180859 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.180818 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.180981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.180959 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 07:03:00.181148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.181134 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k8zmf\"" Jul 09 07:03:00.181234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.181136 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 07:03:00.182656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.182620 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 07:03:00.182955 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.182919 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 07:03:00.184740 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.184554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kvbwn\"" Jul 09 07:03:00.184740 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.184680 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 07:03:00.191638 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.191619 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g4vhc"] Jul 09 07:03:00.192650 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.192086 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pv68j"] Jul 09 07:03:00.285635 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285597 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e2479eae-855f-4c16-b0e8-23b6bff78e0f-config-volume\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.285808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.285808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.285808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f99831a5-a724-478f-8146-181ea93651c5-cert\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.285808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.286019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285825 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.286019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2479eae-855f-4c16-b0e8-23b6bff78e0f-tmp-dir\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.286019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.286019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285922 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.286019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.285967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lf7q\" (UniqueName: \"kubernetes.io/projected/f99831a5-a724-478f-8146-181ea93651c5-kube-api-access-7lf7q\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.286197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.286026 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knr6k\" (UniqueName: \"kubernetes.io/projected/e2479eae-855f-4c16-b0e8-23b6bff78e0f-kube-api-access-knr6k\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.286197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.286078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf2fj\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.286197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.286121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.286197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.286165 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.386967 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.386927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e2479eae-855f-4c16-b0e8-23b6bff78e0f-config-volume\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.386985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387057 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f99831a5-a724-478f-8146-181ea93651c5-cert\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.387134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387115 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2479eae-855f-4c16-b0e8-23b6bff78e0f-tmp-dir\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387184 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lf7q\" (UniqueName: \"kubernetes.io/projected/f99831a5-a724-478f-8146-181ea93651c5-kube-api-access-7lf7q\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knr6k\" (UniqueName: \"kubernetes.io/projected/e2479eae-855f-4c16-b0e8-23b6bff78e0f-kube-api-access-knr6k\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf2fj\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387426 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.387342 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:00.387681 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.387448 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:00.887425056 +0000 UTC m=+35.561281900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:00.387681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2479eae-855f-4c16-b0e8-23b6bff78e0f-tmp-dir\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387657 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387679 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e2479eae-855f-4c16-b0e8-23b6bff78e0f-config-volume\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.387867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.387773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.387946 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.387872 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:00.387946 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.387887 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:00.387946 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.387942 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:00.887930285 +0000 UTC m=+35.561787110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:00.388837 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.388792 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.391622 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.391600 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.391737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.391600 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.391737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.391627 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f99831a5-a724-478f-8146-181ea93651c5-cert\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.401102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.401073 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lf7q\" (UniqueName: \"kubernetes.io/projected/f99831a5-a724-478f-8146-181ea93651c5-kube-api-access-7lf7q\") pod \"ingress-canary-g4vhc\" (UID: \"f99831a5-a724-478f-8146-181ea93651c5\") " pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.401241 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.401161 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knr6k\" (UniqueName: \"kubernetes.io/projected/e2479eae-855f-4c16-b0e8-23b6bff78e0f-kube-api-access-knr6k\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.401553 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.401530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf2fj\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.401828 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.401807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.500320 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.500282 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g4vhc" Jul 09 07:03:00.649265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.649011 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g4vhc"] Jul 09 07:03:00.654419 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:00.654377 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99831a5_a724_478f_8146_181ea93651c5.slice/crio-49257de885349f8a5f60fcd5c699a3b77a7bb7e5bed2700ea851f161d774ee6e WatchSource:0}: Error finding container 49257de885349f8a5f60fcd5c699a3b77a7bb7e5bed2700ea851f161d774ee6e: Status 404 returned error can't find the container with id 49257de885349f8a5f60fcd5c699a3b77a7bb7e5bed2700ea851f161d774ee6e Jul 09 07:03:00.784067 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.784026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:00.881602 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.881572 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:00.891627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.891601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:00.891771 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.891659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:00.891771 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.891716 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:00.891771 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.891759 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:00.891771 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.891769 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:00.891913 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.891783 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:01.891768943 +0000 UTC m=+36.565625771 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:00.891913 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:00.891838 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:01.891803725 +0000 UTC m=+36.565660550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:00.912199 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.912131 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:03:00.912339 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.912131 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:03:00.912430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.912131 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:03:00.914416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914370 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 07:03:00.914416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914409 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 07:03:00.914612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914409 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6ktp4\"" Jul 09 07:03:00.914612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914442 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 07:03:00.914612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914409 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h62pq\"" Jul 09 07:03:00.914612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:00.914379 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 07:03:01.072966 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:01.072923 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g4vhc" event={"ID":"f99831a5-a724-478f-8146-181ea93651c5","Type":"ContainerStarted","Data":"49257de885349f8a5f60fcd5c699a3b77a7bb7e5bed2700ea851f161d774ee6e"} Jul 09 07:03:01.900521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:01.900485 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:01.900569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:01.900654 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:01.900667 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:01.900678 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:01.900741 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:03.900723615 +0000 UTC m=+38.574580444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:01.900958 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:01.900765 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:03.900751876 +0000 UTC m=+38.574608718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:02.840624 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:02.840584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:02.942694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:02.942663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:03.040983 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:03.040951 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:03.083915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:03.083855 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerStarted","Data":"82b04d249a93300a69b211733859a2ffe1c1f45553f8d72efd253c3c2e8ffa0c"} Jul 09 07:03:03.115011 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:03.114961 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:03.917640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:03.917604 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:03.917835 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:03.917674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:03.917835 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:03.917769 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:03.917835 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:03.917787 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:03.917835 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:03.917795 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:03.918034 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:03.917847 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:07.917830806 +0000 UTC m=+42.591687653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:03.918034 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:03.917869 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:07.917859182 +0000 UTC m=+42.591716020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:04.089430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:04.089369 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="82b04d249a93300a69b211733859a2ffe1c1f45553f8d72efd253c3c2e8ffa0c" exitCode=0 Jul 09 07:03:04.089842 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:04.089458 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"82b04d249a93300a69b211733859a2ffe1c1f45553f8d72efd253c3c2e8ffa0c"} Jul 09 07:03:04.166987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:04.166950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:04.874513 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:04.874247 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:04.975894 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:04.975826 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:05.094993 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:05.094963 2573 generic.go:356] "Generic (PLEG): container finished" podID="74999f2c-f351-4a55-ba48-337671bc61df" containerID="741c7d8a7883566bcd61c53bd493ee48f80d3b15081e920baffeab654d889b67" exitCode=0 Jul 09 07:03:05.095442 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:05.095050 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerDied","Data":"741c7d8a7883566bcd61c53bd493ee48f80d3b15081e920baffeab654d889b67"} Jul 09 07:03:05.096641 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:05.096620 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g4vhc" event={"ID":"f99831a5-a724-478f-8146-181ea93651c5","Type":"ContainerStarted","Data":"5de6109fe52763cdc8096e616842d96946d771b5344c2e1d86b86ef496864b32"} Jul 09 07:03:05.131697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:05.131650 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g4vhc" podStartSLOduration=0.909414011 podStartE2EDuration="5.131635193s" totalImagesPullingTime="4.222221182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:05.131060592 +0000 UTC m=+39.804917463" watchObservedRunningTime="2026-07-09 07:03:05.131635193 +0000 UTC m=+39.805492060" Jul 09 07:03:06.102834 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:06.102797 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" event={"ID":"74999f2c-f351-4a55-ba48-337671bc61df","Type":"ContainerStarted","Data":"152308c29b9c2d0d3d47132812b1da681dabefc37d9441e115480cad97965ac6"} Jul 09 07:03:06.122796 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:06.122738 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bvzsl" podStartSLOduration=8.272427707 podStartE2EDuration="40.122721579s" totalImagesPullingTime="31.850293872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:06.122227172 +0000 UTC m=+40.796084016" watchObservedRunningTime="2026-07-09 07:03:06.122721579 +0000 UTC m=+40.796578426" Jul 09 07:03:06.242039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:06.242006 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:06.298923 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:06.298891 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:07.951198 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:07.951163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:07.951223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:07.951301 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:07.951332 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:07.951342 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:07.951362 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:15.9513461 +0000 UTC m=+50.625202925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:07.951576 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:07.951386 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:15.951375895 +0000 UTC m=+50.625232720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:08.647856 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:08.647821 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:08.748242 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:08.748207 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:11.955559 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:11.955523 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.009538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.009513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.200023 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.199989 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.299445 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.299374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.396093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.396053 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.456030 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.456001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.502754 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.502723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.658280 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.658252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.759544 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.759508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.890972 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.890939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:12.991875 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:12.991800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:13.123979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.123953 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jd546"] Jul 09 07:03:13.159217 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.159189 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jd546"] Jul 09 07:03:13.159385 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.159334 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.161662 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.161633 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 07:03:13.161662 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.161649 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 07:03:13.161833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.161638 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 07:03:13.161833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.161655 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rfq8r\"" Jul 09 07:03:13.161833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.161638 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 07:03:13.167353 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.167334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:13.187231 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.187212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-crio-socket\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.187357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.187273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.187357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.187331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.187485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.187417 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624qx\" (UniqueName: \"kubernetes.io/projected/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-api-access-624qx\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.187485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.187454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-data-volume\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288169 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-624qx\" (UniqueName: \"kubernetes.io/projected/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-api-access-624qx\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-data-volume\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288355 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.288226 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 07:03:13.288355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-crio-socket\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288355 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.288300 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:13.788280027 +0000 UTC m=+48.462136870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : configmap "kube-rbac-proxy" not found Jul 09 07:03:13.288529 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288529 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288414 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-crio-socket\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.288529 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.288511 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 07:03:13.288621 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.288558 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:13.788545221 +0000 UTC m=+48.462402050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : secret "insights-runtime-extractor-tls" not found Jul 09 07:03:13.288621 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.288580 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-data-volume\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.298492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.298472 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-624qx\" (UniqueName: \"kubernetes.io/projected/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-api-access-624qx\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.791215 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.791179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.791377 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:13.791228 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:13.791377 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.791315 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 07:03:13.791377 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.791325 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 07:03:13.791377 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.791373 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:14.791360291 +0000 UTC m=+49.465217115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : configmap "kube-rbac-proxy" not found Jul 09 07:03:13.791545 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:13.791387 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:14.791380196 +0000 UTC m=+49.465237022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : secret "insights-runtime-extractor-tls" not found Jul 09 07:03:14.407577 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:14.407546 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:14.512158 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:14.512116 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:14.798697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:14.798620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:14.798828 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:14.798727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:14.798828 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:14.798751 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 07:03:14.798828 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:14.798822 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:16.798802683 +0000 UTC m=+51.472659527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : configmap "kube-rbac-proxy" not found Jul 09 07:03:14.798933 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:14.798849 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 07:03:14.798933 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:14.798902 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:16.798890391 +0000 UTC m=+51.472747215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : secret "insights-runtime-extractor-tls" not found Jul 09 07:03:15.139452 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:15.139413 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:16.009044 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.009005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.009079 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.009172 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.009203 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.009222 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls podName:e2479eae-855f-4c16-b0e8-23b6bff78e0f nodeName:}" failed. No retries permitted until 2026-07-09 07:03:32.00921025 +0000 UTC m=+66.683067075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls") pod "dns-default-pv68j" (UID: "e2479eae-855f-4c16-b0e8-23b6bff78e0f") : secret "dns-default-metrics-tls" not found Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.009227 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-677fcdd584-vkzb9: secret "image-registry-tls" not found Jul 09 07:03:16.009522 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.009297 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls podName:006a6360-e47e-460d-8c7a-db99d295a547 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:32.009277879 +0000 UTC m=+66.683134728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls") pod "image-registry-677fcdd584-vkzb9" (UID: "006a6360-e47e-460d-8c7a-db99d295a547") : secret "image-registry-tls" not found Jul 09 07:03:16.109925 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.109899 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztdwp_d977d649-07ee-4d4b-b64e-3f89c121db76/dns-node-resolver/0.log" Jul 09 07:03:16.425522 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.425494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:16.526226 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.526191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:16.710620 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.710549 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dnh77_0b07514b-c2af-4d17-99a6-37e4102ac780/node-ca/0.log" Jul 09 07:03:16.814620 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.814586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:16.814779 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:16.814630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:16.814779 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.814712 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 07:03:16.814779 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.814724 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 07:03:16.814779 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.814760 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:20.814746505 +0000 UTC m=+55.488603330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : configmap "kube-rbac-proxy" not found Jul 09 07:03:16.814779 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:16.814772 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:20.814766103 +0000 UTC m=+55.488622928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : secret "insights-runtime-extractor-tls" not found Jul 09 07:03:17.710433 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:17.710359 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g4vhc_f99831a5-a724-478f-8146-181ea93651c5/serve-healthcheck-canary/0.log" Jul 09 07:03:17.843992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:17.843958 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:17.997723 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:17.997648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.098653 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.098602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.198835 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.198807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.298586 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.298504 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.415376 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.415341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.517637 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.517604 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:18.654400 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:18.654377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:19.535040 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:19.535010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:19.941524 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:19.941490 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:20.133784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:20.133751 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:20.335331 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:20.335255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:20.734927 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:20.734895 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:20.847716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:20.847677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:20.847907 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:20.847734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:20.847907 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:20.847820 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 07:03:20.847907 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:20.847869 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 07:03:20.847907 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:20.847881 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:28.847866419 +0000 UTC m=+63.521723244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : secret "insights-runtime-extractor-tls" not found Jul 09 07:03:20.848071 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:20.847934 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm podName:3e3b2de8-ae2b-4aaa-949a-fed1854767f2 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:28.847918718 +0000 UTC m=+63.521775545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jd546" (UID: "3e3b2de8-ae2b-4aaa-949a-fed1854767f2") : configmap "kube-rbac-proxy" not found Jul 09 07:03:21.533051 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:21.533020 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:21.651235 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:21.651205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:21.750792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:21.750755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:21.801466 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:21.801376 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:22.137940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:22.137916 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:22.336740 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:22.336709 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:22.390597 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:22.390534 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:22.933647 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:22.933617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:23.541716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:23.541685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:23.737266 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:23.737230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:24.134467 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:24.134439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:24.234982 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:24.234951 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:24.385138 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:24.385058 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:24.937705 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:24.937664 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:25.338514 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:25.338435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:25.439234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:25.439198 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:25.742100 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:25.742074 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:26.135106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:26.135075 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:26.410905 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:26.410829 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:26.544661 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:26.544622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:26.933200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:26.933170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:27.075275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:27.075249 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzjwl" Jul 09 07:03:27.338536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:27.338461 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:27.542657 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:27.542625 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:27.938730 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:27.938701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:28.335048 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.334965 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:28.539754 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.539722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:28.912682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.912645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:28.913067 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.912695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:28.913166 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.913150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:28.914941 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:28.914917 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3b2de8-ae2b-4aaa-949a-fed1854767f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jd546\" (UID: \"3e3b2de8-ae2b-4aaa-949a-fed1854767f2\") " pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:29.070654 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:29.070617 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rfq8r\"" Jul 09 07:03:29.079333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:29.079309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jd546" Jul 09 07:03:29.198841 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:29.198740 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jd546"] Jul 09 07:03:29.222886 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:29.222854 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3b2de8_ae2b_4aaa_949a_fed1854767f2.slice/crio-ff2b3a9b62d20b8450f1e15fae21ae82f9450def1f32578e7224d3672d031011 WatchSource:0}: Error finding container ff2b3a9b62d20b8450f1e15fae21ae82f9450def1f32578e7224d3672d031011: Status 404 returned error can't find the container with id ff2b3a9b62d20b8450f1e15fae21ae82f9450def1f32578e7224d3672d031011 Jul 09 07:03:30.160610 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:30.160542 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jd546" event={"ID":"3e3b2de8-ae2b-4aaa-949a-fed1854767f2","Type":"ContainerStarted","Data":"ff2b3a9b62d20b8450f1e15fae21ae82f9450def1f32578e7224d3672d031011"} Jul 09 07:03:31.535135 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.535088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:03:31.537459 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.537439 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 07:03:31.548325 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.548295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19428e47-7c22-4917-a531-e27501b88251-original-pull-secret\") pod \"global-pull-secret-syncer-4w92n\" (UID: \"19428e47-7c22-4917-a531-e27501b88251\") " pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:03:31.635723 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.635685 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:03:31.635723 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.635733 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:03:31.638180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.638025 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 07:03:31.638180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.638124 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 07:03:31.648678 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.648653 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 07:03:31.649321 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.649262 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf17b4e-bde3-42fc-9019-eaa60ea69b84-metrics-certs\") pod \"network-metrics-daemon-gvzjc\" (UID: \"acf17b4e-bde3-42fc-9019-eaa60ea69b84\") " pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:03:31.660308 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.660278 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd45r\" (UniqueName: \"kubernetes.io/projected/4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f-kube-api-access-wd45r\") pod \"network-check-target-vf8f8\" (UID: \"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f\") " pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:03:31.822865 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.822780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4w92n" Jul 09 07:03:31.830062 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.830036 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6ktp4\"" Jul 09 07:03:31.833536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.833508 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h62pq\"" Jul 09 07:03:31.838667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.838642 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:03:31.842078 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:31.842057 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvzjc" Jul 09 07:03:32.039845 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.039796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:32.040024 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.039869 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:32.042491 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.042426 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2479eae-855f-4c16-b0e8-23b6bff78e0f-metrics-tls\") pod \"dns-default-pv68j\" (UID: \"e2479eae-855f-4c16-b0e8-23b6bff78e0f\") " pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:32.042825 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.042797 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"image-registry-677fcdd584-vkzb9\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:32.279997 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.279965 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z7kjk\"" Jul 09 07:03:32.288670 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.288648 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:32.294777 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.294755 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k8zmf\"" Jul 09 07:03:32.303144 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:32.303121 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:33.001279 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.001238 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvzjc"] Jul 09 07:03:33.031855 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.031823 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:33.159358 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.159274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:33.235196 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:33.235145 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf17b4e_bde3_42fc_9019_eaa60ea69b84.slice/crio-a41593fe767ad1ab791d9b85d2626d6dc8f43aad60aa24e55e8cf8e6626ae750 WatchSource:0}: Error finding container a41593fe767ad1ab791d9b85d2626d6dc8f43aad60aa24e55e8cf8e6626ae750: Status 404 returned error can't find the container with id a41593fe767ad1ab791d9b85d2626d6dc8f43aad60aa24e55e8cf8e6626ae750 Jul 09 07:03:33.381849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.381822 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pv68j"] Jul 09 07:03:33.396585 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.396558 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4w92n"] Jul 09 07:03:33.460469 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:33.460380 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2479eae_855f_4c16_b0e8_23b6bff78e0f.slice/crio-d0d322630e75c91953cb7bf7293d4b072b9863227c5305ad6d9fe8b9e83ef3ef WatchSource:0}: Error finding container d0d322630e75c91953cb7bf7293d4b072b9863227c5305ad6d9fe8b9e83ef3ef: Status 404 returned error can't find the container with id d0d322630e75c91953cb7bf7293d4b072b9863227c5305ad6d9fe8b9e83ef3ef Jul 09 07:03:33.461183 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:33.461163 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19428e47_7c22_4917_a531_e27501b88251.slice/crio-ffb2651b40b836e0668291c3cee8da8a3907708b20197bb81b5b0a033456c003 WatchSource:0}: Error finding container ffb2651b40b836e0668291c3cee8da8a3907708b20197bb81b5b0a033456c003: Status 404 returned error can't find the container with id ffb2651b40b836e0668291c3cee8da8a3907708b20197bb81b5b0a033456c003 Jul 09 07:03:33.614774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.614741 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vf8f8"] Jul 09 07:03:33.618594 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:33.618568 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c98d42b_c3e9_4e6d_9e4e_3f3bb2ff777f.slice/crio-5575b530bc3945b2ea4d6056772c0d83c884840aef39fa3ec46e44ec73517b56 WatchSource:0}: Error finding container 5575b530bc3945b2ea4d6056772c0d83c884840aef39fa3ec46e44ec73517b56: Status 404 returned error can't find the container with id 5575b530bc3945b2ea4d6056772c0d83c884840aef39fa3ec46e44ec73517b56 Jul 09 07:03:33.618981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:33.618955 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:03:33.622626 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:33.622598 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006a6360_e47e_460d_8c7a_db99d295a547.slice/crio-4f64b73281d4461a5e4ccb012aaa0da1b597310595f5a54660cc8e1459a20714 WatchSource:0}: Error finding container 4f64b73281d4461a5e4ccb012aaa0da1b597310595f5a54660cc8e1459a20714: Status 404 returned error can't find the container with id 4f64b73281d4461a5e4ccb012aaa0da1b597310595f5a54660cc8e1459a20714 Jul 09 07:03:34.174683 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.174645 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jd546" event={"ID":"3e3b2de8-ae2b-4aaa-949a-fed1854767f2","Type":"ContainerStarted","Data":"f5da634b7bc82e403de88259497153463ea18e394d3692d6a900bf9edaa9037a"} Jul 09 07:03:34.176981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.176945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pv68j" event={"ID":"e2479eae-855f-4c16-b0e8-23b6bff78e0f","Type":"ContainerStarted","Data":"d0d322630e75c91953cb7bf7293d4b072b9863227c5305ad6d9fe8b9e83ef3ef"} Jul 09 07:03:34.178476 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.178448 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vf8f8" event={"ID":"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f","Type":"ContainerStarted","Data":"5575b530bc3945b2ea4d6056772c0d83c884840aef39fa3ec46e44ec73517b56"} Jul 09 07:03:34.180167 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.180134 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvzjc" event={"ID":"acf17b4e-bde3-42fc-9019-eaa60ea69b84","Type":"ContainerStarted","Data":"a41593fe767ad1ab791d9b85d2626d6dc8f43aad60aa24e55e8cf8e6626ae750"} Jul 09 07:03:34.182509 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.182482 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4w92n" event={"ID":"19428e47-7c22-4917-a531-e27501b88251","Type":"ContainerStarted","Data":"ffb2651b40b836e0668291c3cee8da8a3907708b20197bb81b5b0a033456c003"} Jul 09 07:03:34.186111 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.186084 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" event={"ID":"006a6360-e47e-460d-8c7a-db99d295a547","Type":"ContainerStarted","Data":"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b"} Jul 09 07:03:34.186212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.186132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" event={"ID":"006a6360-e47e-460d-8c7a-db99d295a547","Type":"ContainerStarted","Data":"4f64b73281d4461a5e4ccb012aaa0da1b597310595f5a54660cc8e1459a20714"} Jul 09 07:03:34.186272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.186249 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:34.205850 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.205794 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" podStartSLOduration=72.205772493 podStartE2EDuration="1m12.205772493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:34.205414333 +0000 UTC m=+68.879271210" watchObservedRunningTime="2026-07-09 07:03:34.205772493 +0000 UTC m=+68.879629345" Jul 09 07:03:34.751892 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.751854 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:03:34.765014 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.764988 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g"] Jul 09 07:03:34.777479 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.777452 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm"] Jul 09 07:03:34.778242 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.777745 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.780775 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.780749 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 07:03:34.781443 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.780971 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 07:03:34.781443 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.781084 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 07:03:34.781443 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.781260 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 07:03:34.781443 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.781415 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bb47f\"" Jul 09 07:03:34.800339 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.800308 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g"] Jul 09 07:03:34.800339 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.800347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm"] Jul 09 07:03:34.800559 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.800453 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.802536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.802516 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 07:03:34.854280 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.854249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:34.862496 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.862466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.862658 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.862522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37412f0d-47e1-4bed-860e-56db9070bbb2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.862658 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.862593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbh8\" (UniqueName: \"kubernetes.io/projected/37412f0d-47e1-4bed-860e-56db9070bbb2-kube-api-access-lqbh8\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.862658 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.862617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-tmp\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.862822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.862697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjjd\" (UniqueName: \"kubernetes.io/projected/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-kube-api-access-sqjjd\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.892263 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.892232 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d67c8b49f-grbrn"] Jul 09 07:03:34.913687 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.913667 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.923556 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.923491 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d67c8b49f-grbrn"] Jul 09 07:03:34.963048 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbh8\" (UniqueName: \"kubernetes.io/projected/37412f0d-47e1-4bed-860e-56db9070bbb2-kube-api-access-lqbh8\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.963169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-image-registry-private-configuration\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-certificates\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963127 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-tmp\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.963169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-installation-pull-secrets\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963187 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-trusted-ca\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37412f0d-47e1-4bed-860e-56db9070bbb2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963299 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjjd\" (UniqueName: \"kubernetes.io/projected/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-kube-api-access-sqjjd\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963326 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-tls\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963351 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0e28daf2-4c8b-463f-862a-9c72ee91a807-ca-trust-extracted\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963665 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-bound-sa-token\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.963665 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.963421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bghz\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-kube-api-access-7bghz\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:34.964387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.964361 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:34.964501 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.964474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-tmp\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.967012 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.966965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37412f0d-47e1-4bed-860e-56db9070bbb2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:34.969809 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.969764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.979494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.979448 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjjd\" (UniqueName: \"kubernetes.io/projected/08f6dd85-ec45-4537-bb30-f2fb070cd7ef-kube-api-access-sqjjd\") pod \"klusterlet-addon-workmgr-c9d4d4579-27scm\" (UID: \"08f6dd85-ec45-4537-bb30-f2fb070cd7ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:34.979599 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:34.979540 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbh8\" (UniqueName: \"kubernetes.io/projected/37412f0d-47e1-4bed-860e-56db9070bbb2-kube-api-access-lqbh8\") pod \"managed-serviceaccount-addon-agent-675d7c4b44-ssl9g\" (UID: \"37412f0d-47e1-4bed-860e-56db9070bbb2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:35.039682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.039653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:35.064773 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064746 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-installation-pull-secrets\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.064906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-trusted-ca\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.064906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064847 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-tls\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.064906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0e28daf2-4c8b-463f-862a-9c72ee91a807-ca-trust-extracted\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.064906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064895 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-bound-sa-token\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.065079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bghz\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-kube-api-access-7bghz\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.065079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-image-registry-private-configuration\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.065079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.064988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-certificates\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.065491 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.065465 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0e28daf2-4c8b-463f-862a-9c72ee91a807-ca-trust-extracted\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.065900 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.065876 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-certificates\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.066306 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.066279 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e28daf2-4c8b-463f-862a-9c72ee91a807-trusted-ca\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.068418 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.068357 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-installation-pull-secrets\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.069351 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.069325 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0e28daf2-4c8b-463f-862a-9c72ee91a807-image-registry-private-configuration\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.069481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.069414 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-registry-tls\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.077235 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.077187 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-bound-sa-token\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.077488 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.077451 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bghz\" (UniqueName: \"kubernetes.io/projected/0e28daf2-4c8b-463f-862a-9c72ee91a807-kube-api-access-7bghz\") pod \"image-registry-6d67c8b49f-grbrn\" (UID: \"0e28daf2-4c8b-463f-862a-9c72ee91a807\") " pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.102346 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.102321 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" Jul 09 07:03:35.112579 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.112536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:35.137377 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.137299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:35.198564 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.198341 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jd546" event={"ID":"3e3b2de8-ae2b-4aaa-949a-fed1854767f2","Type":"ContainerStarted","Data":"892dee30f6e7e29875cb2b88573a1a69159a9e596bbf8bf66920b0107391d35d"} Jul 09 07:03:35.201755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.201668 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvzjc" event={"ID":"acf17b4e-bde3-42fc-9019-eaa60ea69b84","Type":"ContainerStarted","Data":"da6a6f43a4f2fb04838ee100345ee32ddabee88804b76ae2777a2af792ad0954"} Jul 09 07:03:35.226279 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.225938 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:35.298788 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.298467 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g"] Jul 09 07:03:35.320049 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.319738 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm"] Jul 09 07:03:35.414338 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:35.414257 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d67c8b49f-grbrn"] Jul 09 07:03:36.019630 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:36.019594 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e28daf2_4c8b_463f_862a_9c72ee91a807.slice/crio-f936ac1ae403a9201b0e3f2606ecd16097b641730f5ea507ae938ebd6dd4f670 WatchSource:0}: Error finding container f936ac1ae403a9201b0e3f2606ecd16097b641730f5ea507ae938ebd6dd4f670: Status 404 returned error can't find the container with id f936ac1ae403a9201b0e3f2606ecd16097b641730f5ea507ae938ebd6dd4f670 Jul 09 07:03:36.020767 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:36.020668 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37412f0d_47e1_4bed_860e_56db9070bbb2.slice/crio-e2fdf4bab941c063026cb0ac846ab38f36707f0d0ca3b3b362c7e1fb3196bdbb WatchSource:0}: Error finding container e2fdf4bab941c063026cb0ac846ab38f36707f0d0ca3b3b362c7e1fb3196bdbb: Status 404 returned error can't find the container with id e2fdf4bab941c063026cb0ac846ab38f36707f0d0ca3b3b362c7e1fb3196bdbb Jul 09 07:03:36.209379 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.209338 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvzjc" event={"ID":"acf17b4e-bde3-42fc-9019-eaa60ea69b84","Type":"ContainerStarted","Data":"1b7cb37f74644d2e505a2222f0cf3bbf1ade43548ead06d865f799cd6c3b8829"} Jul 09 07:03:36.211796 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.211695 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" event={"ID":"0e28daf2-4c8b-463f-862a-9c72ee91a807","Type":"ContainerStarted","Data":"7d561dc40a1910e4795f2135bd320840c07b9f2462f6bf90f503495827dbf36f"} Jul 09 07:03:36.211796 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.211741 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" event={"ID":"0e28daf2-4c8b-463f-862a-9c72ee91a807","Type":"ContainerStarted","Data":"f936ac1ae403a9201b0e3f2606ecd16097b641730f5ea507ae938ebd6dd4f670"} Jul 09 07:03:36.213332 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.212304 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:36.214537 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.214450 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" event={"ID":"08f6dd85-ec45-4537-bb30-f2fb070cd7ef","Type":"ContainerStarted","Data":"e28b4be0fd10895c58648c727aac2b9e679539d0e8a2fd980c16081e5513523a"} Jul 09 07:03:36.216212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.216184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" event={"ID":"37412f0d-47e1-4bed-860e-56db9070bbb2","Type":"ContainerStarted","Data":"e2fdf4bab941c063026cb0ac846ab38f36707f0d0ca3b3b362c7e1fb3196bdbb"} Jul 09 07:03:36.248254 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.247144 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gvzjc" podStartSLOduration=68.591860972 podStartE2EDuration="1m10.247053509s" totalImagesPullingTime="1.655192537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:36.227536884 +0000 UTC m=+70.901393744" watchObservedRunningTime="2026-07-09 07:03:36.247053509 +0000 UTC m=+70.920910360" Jul 09 07:03:36.248254 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.247906 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" podStartSLOduration=2.247893467 podStartE2EDuration="2.247893467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:36.244976808 +0000 UTC m=+70.918833678" watchObservedRunningTime="2026-07-09 07:03:36.247893467 +0000 UTC m=+70.921750295" Jul 09 07:03:36.309285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.309198 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:36.414536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.414465 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:36.509067 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:36.509032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:37.226298 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.226147 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pv68j" event={"ID":"e2479eae-855f-4c16-b0e8-23b6bff78e0f","Type":"ContainerStarted","Data":"1b5cdcde26a98556ceefe11bf887aa853bf974a98183122523956e984ed6d734"} Jul 09 07:03:37.226298 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.226199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pv68j" event={"ID":"e2479eae-855f-4c16-b0e8-23b6bff78e0f","Type":"ContainerStarted","Data":"e70c707a6c3ba511c877a585e126636305eafb67eef537f27bab8e0890f90df6"} Jul 09 07:03:37.226298 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.226229 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:37.245063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.244378 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pv68j" podStartSLOduration=34.536862728 podStartE2EDuration="37.244357687s" totalImagesPullingTime="2.707494959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:37.24355434 +0000 UTC m=+71.917411226" watchObservedRunningTime="2026-07-09 07:03:37.244357687 +0000 UTC m=+71.918214516" Jul 09 07:03:37.784123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.783983 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:37.888256 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.888181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:37.982714 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:37.982640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.043046 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.042927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.061895 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.061858 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb"] Jul 09 07:03:38.063992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.063969 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:38.066131 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.066105 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 07:03:38.066231 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.066136 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xdlkw\"" Jul 09 07:03:38.071742 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.071703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb"] Jul 09 07:03:38.137181 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.137134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.192674 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.192632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5963dd1d-5bed-4dc7-8249-a8c3a08e1320-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h58qb\" (UID: \"5963dd1d-5bed-4dc7-8249-a8c3a08e1320\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:38.234493 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.234454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.293530 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.293461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5963dd1d-5bed-4dc7-8249-a8c3a08e1320-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h58qb\" (UID: \"5963dd1d-5bed-4dc7-8249-a8c3a08e1320\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:38.296052 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.296025 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5963dd1d-5bed-4dc7-8249-a8c3a08e1320-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h58qb\" (UID: \"5963dd1d-5bed-4dc7-8249-a8c3a08e1320\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:38.335583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.335506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.377212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.377169 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:38.452322 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.452274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:38.551765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:38.551684 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:39.271272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:39.271180 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:39.326986 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:39.326950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:39.649265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:39.649229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:39.749227 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:39.749191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:39.850868 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:39.850836 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:40.054145 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:40.054042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:41.068155 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:41.068119 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:41.168817 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:41.168775 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:41.219787 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:41.219706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:41.666685 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:41.666650 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:41.721723 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:41.721690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:43.224784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.224711 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb"] Jul 09 07:03:43.230734 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:43.230688 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5963dd1d_5bed_4dc7_8249_a8c3a08e1320.slice/crio-b62cc90e0bd6e860fe54350b102b0ab1a55c676c0906d74d08756d0fb9955bb5 WatchSource:0}: Error finding container b62cc90e0bd6e860fe54350b102b0ab1a55c676c0906d74d08756d0fb9955bb5: Status 404 returned error can't find the container with id b62cc90e0bd6e860fe54350b102b0ab1a55c676c0906d74d08756d0fb9955bb5 Jul 09 07:03:43.250119 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.249707 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jd546" event={"ID":"3e3b2de8-ae2b-4aaa-949a-fed1854767f2","Type":"ContainerStarted","Data":"0577acac5720499ea0d2af74bd20a8e68fccaf53323db6d449e32df1c32d7426"} Jul 09 07:03:43.251564 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.251514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vf8f8" event={"ID":"4c98d42b-c3e9-4e6d-9e4e-3f3bb2ff777f","Type":"ContainerStarted","Data":"72db2418a9195a0bbf4448a850347b996fe2d5547eb71caeb6a7d628f21addf9"} Jul 09 07:03:43.251696 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.251579 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:03:43.254558 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.254484 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" event={"ID":"5963dd1d-5bed-4dc7-8249-a8c3a08e1320","Type":"ContainerStarted","Data":"b62cc90e0bd6e860fe54350b102b0ab1a55c676c0906d74d08756d0fb9955bb5"} Jul 09 07:03:43.257202 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.256804 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" event={"ID":"08f6dd85-ec45-4537-bb30-f2fb070cd7ef","Type":"ContainerStarted","Data":"9b7086c4800c6cbe99680e1a4c92572d265ff1a1370b0b6a348ea95e657022d7"} Jul 09 07:03:43.257202 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.257163 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:43.258277 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.258240 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" podUID="08f6dd85-ec45-4537-bb30-f2fb070cd7ef" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.11:8000/readyz\": dial tcp 10.134.0.11:8000: connect: connection refused" Jul 09 07:03:43.259177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.259148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" event={"ID":"37412f0d-47e1-4bed-860e-56db9070bbb2","Type":"ContainerStarted","Data":"9f5ed11015424d743c3106e7b4a34b7948e1710481c61cab1f22fffb70c2c612"} Jul 09 07:03:43.268429 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.268205 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jd546" podStartSLOduration=16.711678599 podStartE2EDuration="30.268190967s" totalImagesPullingTime="13.556512368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:03:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:43.267608836 +0000 UTC m=+77.941465730" watchObservedRunningTime="2026-07-09 07:03:43.268190967 +0000 UTC m=+77.942047813" Jul 09 07:03:43.281562 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.281511 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vf8f8" podStartSLOduration=67.812092781 podStartE2EDuration="1m17.28149289s" totalImagesPullingTime="9.469400109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:43.281076911 +0000 UTC m=+77.954933761" watchObservedRunningTime="2026-07-09 07:03:43.28149289 +0000 UTC m=+77.955349740" Jul 09 07:03:43.297292 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.297227 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-675d7c4b44-ssl9g" podStartSLOduration=2.175992193 podStartE2EDuration="9.297206238s" totalImagesPullingTime="7.121214045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:43.296370101 +0000 UTC m=+77.970226949" watchObservedRunningTime="2026-07-09 07:03:43.297206238 +0000 UTC m=+77.971063086" Jul 09 07:03:43.310247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:43.310180 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" podStartSLOduration=2.206276248 podStartE2EDuration="9.309842868s" totalImagesPullingTime="7.10356662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:43.30919491 +0000 UTC m=+77.983051780" watchObservedRunningTime="2026-07-09 07:03:43.309842868 +0000 UTC m=+77.983699727" Jul 09 07:03:44.264583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:44.264547 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4w92n" event={"ID":"19428e47-7c22-4917-a531-e27501b88251","Type":"ContainerStarted","Data":"66e2723a2aa77ec473d15dbd99049055b33391d37142fa4d1d85ad78082ed4e5"} Jul 09 07:03:44.266410 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:44.266366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c9d4d4579-27scm" Jul 09 07:03:44.278212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:44.278158 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4w92n" podStartSLOduration=68.63430301 podStartE2EDuration="1m18.278139091s" totalImagesPullingTime="9.643836081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:44.277606129 +0000 UTC m=+78.951462995" watchObservedRunningTime="2026-07-09 07:03:44.278139091 +0000 UTC m=+78.951995933" Jul 09 07:03:45.269069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:45.269033 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" event={"ID":"5963dd1d-5bed-4dc7-8249-a8c3a08e1320","Type":"ContainerStarted","Data":"ad77ed231851e3c9c76f1b9316159dd516cf8cbb14c7f0daca04b4b480fa6991"} Jul 09 07:03:45.283192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:45.283147 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" podStartSLOduration=6.040239691 podStartE2EDuration="7.283132469s" totalImagesPullingTime="1.242892778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:45.281980447 +0000 UTC m=+79.955837295" watchObservedRunningTime="2026-07-09 07:03:45.283132469 +0000 UTC m=+79.956989316" Jul 09 07:03:46.273302 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:46.273269 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:46.277667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:46.277646 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h58qb" Jul 09 07:03:46.281021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:46.280978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:46.372277 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:46.372235 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:46.995068 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:46.995030 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:47.029735 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.029709 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf"] Jul 09 07:03:47.032637 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.032620 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.034884 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.034858 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 07:03:47.034884 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.034874 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 07:03:47.035046 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.034899 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p5sk4\"" Jul 09 07:03:47.035542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.035525 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 07:03:47.035612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.035543 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 07:03:47.035655 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.035524 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 07:03:47.043310 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.043286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf"] Jul 09 07:03:47.060614 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.060586 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:47.162223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.162088 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdfd242-9d27-494b-b2e6-719951e8ad53-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.162223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.162144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnxp2\" (UniqueName: \"kubernetes.io/projected/3fdfd242-9d27-494b-b2e6-719951e8ad53-kube-api-access-vnxp2\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.162223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.162195 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.162223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.162217 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.232032 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.232005 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pv68j" Jul 09 07:03:47.263142 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.263066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.263142 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.263097 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.263142 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.263130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdfd242-9d27-494b-b2e6-719951e8ad53-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.263476 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.263153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnxp2\" (UniqueName: \"kubernetes.io/projected/3fdfd242-9d27-494b-b2e6-719951e8ad53-kube-api-access-vnxp2\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.263476 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:47.263323 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 07:03:47.263476 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:03:47.263424 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls podName:3fdfd242-9d27-494b-b2e6-719951e8ad53 nodeName:}" failed. No retries permitted until 2026-07-09 07:03:47.763386555 +0000 UTC m=+82.437243386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-kk5mf" (UID: "3fdfd242-9d27-494b-b2e6-719951e8ad53") : secret "prometheus-operator-tls" not found Jul 09 07:03:47.263848 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.263823 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdfd242-9d27-494b-b2e6-719951e8ad53-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.265849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.265822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.272706 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.272685 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnxp2\" (UniqueName: \"kubernetes.io/projected/3fdfd242-9d27-494b-b2e6-719951e8ad53-kube-api-access-vnxp2\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.319409 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.319353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:47.766038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.765997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.768309 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.768287 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fdfd242-9d27-494b-b2e6-719951e8ad53-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kk5mf\" (UID: \"3fdfd242-9d27-494b-b2e6-719951e8ad53\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:47.941315 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:47.941286 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" Jul 09 07:03:48.082276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:48.082243 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf"] Jul 09 07:03:48.084995 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:48.084969 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdfd242_9d27_494b_b2e6_719951e8ad53.slice/crio-e69db995659f1347554f1df6b386f2971dea73e97a50f5c79e226e1426b45fa1 WatchSource:0}: Error finding container e69db995659f1347554f1df6b386f2971dea73e97a50f5c79e226e1426b45fa1: Status 404 returned error can't find the container with id e69db995659f1347554f1df6b386f2971dea73e97a50f5c79e226e1426b45fa1 Jul 09 07:03:48.281581 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:48.281543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" event={"ID":"3fdfd242-9d27-494b-b2e6-719951e8ad53","Type":"ContainerStarted","Data":"e69db995659f1347554f1df6b386f2971dea73e97a50f5c79e226e1426b45fa1"} Jul 09 07:03:48.470772 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:48.470740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:48.691341 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:48.691309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:49.793496 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:49.793386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:50.291177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:50.291148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" event={"ID":"3fdfd242-9d27-494b-b2e6-719951e8ad53","Type":"ContainerStarted","Data":"180a53161569db9cedd3183aef867cada99ce44ee82675df1154dc2b4516cf91"} Jul 09 07:03:50.291351 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:50.291199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" event={"ID":"3fdfd242-9d27-494b-b2e6-719951e8ad53","Type":"ContainerStarted","Data":"7d082543cedc99f7f92313e5984196c592b50bf757879249c7309890f216abf8"} Jul 09 07:03:50.306995 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:50.306952 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kk5mf" podStartSLOduration=1.858092783 podStartE2EDuration="3.306937388s" totalImagesPullingTime="1.448844605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:50.306009236 +0000 UTC m=+84.979866118" watchObservedRunningTime="2026-07-09 07:03:50.306937388 +0000 UTC m=+84.980794266" Jul 09 07:03:52.139200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.139163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.368887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.368855 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.399945 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.399876 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9wpmm"] Jul 09 07:03:52.403307 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.403292 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.405472 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.405445 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 07:03:52.405472 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.405468 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8kjsj\"" Jul 09 07:03:52.405928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.405907 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 07:03:52.406032 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.405928 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 07:03:52.452610 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.452583 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.501721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501693 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-textfile\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.501721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-tls\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.501919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltbd9\" (UniqueName: \"kubernetes.io/projected/e8f383b6-8045-45f4-a2a3-a50e0674607c-kube-api-access-ltbd9\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.501919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.501919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-sys\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.501919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-root\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.502038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-wtmp\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.502038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.501972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-metrics-client-ca\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.502038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.502001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.506835 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.506810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.603282 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-textfile\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603282 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603283 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-tls\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltbd9\" (UniqueName: \"kubernetes.io/projected/e8f383b6-8045-45f4-a2a3-a50e0674607c-kube-api-access-ltbd9\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603329 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-sys\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603381 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-root\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-wtmp\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-metrics-client-ca\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603512 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-root\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603909 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603595 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-wtmp\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603909 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603650 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-textfile\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.603909 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.603661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f383b6-8045-45f4-a2a3-a50e0674607c-sys\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.604129 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.604108 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-metrics-client-ca\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.604165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.604108 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.605728 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.605705 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-tls\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.605815 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.605755 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8f383b6-8045-45f4-a2a3-a50e0674607c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.607533 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.607515 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.613121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.613095 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltbd9\" (UniqueName: \"kubernetes.io/projected/e8f383b6-8045-45f4-a2a3-a50e0674607c-kube-api-access-ltbd9\") pod \"node-exporter-9wpmm\" (UID: \"e8f383b6-8045-45f4-a2a3-a50e0674607c\") " pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.707702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.707614 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:52.713296 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.713279 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9wpmm" Jul 09 07:03:52.723508 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:52.723484 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f383b6_8045_45f4_a2a3_a50e0674607c.slice/crio-0ae5e6fd008ca4b54dea3a6e54d9ab629f71e60d5fc05f02b559123219a7fe1d WatchSource:0}: Error finding container 0ae5e6fd008ca4b54dea3a6e54d9ab629f71e60d5fc05f02b559123219a7fe1d: Status 404 returned error can't find the container with id 0ae5e6fd008ca4b54dea3a6e54d9ab629f71e60d5fc05f02b559123219a7fe1d Jul 09 07:03:52.817206 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:52.817171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:53.165481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:53.165449 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:53.305123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:53.305088 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wpmm" event={"ID":"e8f383b6-8045-45f4-a2a3-a50e0674607c","Type":"ContainerStarted","Data":"0ae5e6fd008ca4b54dea3a6e54d9ab629f71e60d5fc05f02b559123219a7fe1d"} Jul 09 07:03:53.377913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:53.377876 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:53.468256 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:53.468148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:53.677970 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:53.677933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:54.070463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.070433 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:03:54.073356 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.073340 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.075543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.075520 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9l2ts\"" Jul 09 07:03:54.076231 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076214 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 07:03:54.076540 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076516 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 07:03:54.076540 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076531 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 07:03:54.076704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076581 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 07:03:54.076704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076624 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 07:03:54.076839 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.076823 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 07:03:54.077023 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.077010 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 07:03:54.080297 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.080233 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 07:03:54.085789 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.085769 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:03:54.214460 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnph6\" (UniqueName: \"kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.214919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.214728 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.309888 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.309852 2573 generic.go:356] "Generic (PLEG): container finished" podID="e8f383b6-8045-45f4-a2a3-a50e0674607c" containerID="f3fa0dc8daba6c2e882975cfc1f3bf3614cb9b284bd2ed004ee094b3bd3fe216" exitCode=0 Jul 09 07:03:54.310039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.309896 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wpmm" event={"ID":"e8f383b6-8045-45f4-a2a3-a50e0674607c","Type":"ContainerDied","Data":"f3fa0dc8daba6c2e882975cfc1f3bf3614cb9b284bd2ed004ee094b3bd3fe216"} Jul 09 07:03:54.316981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.316952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317128 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.316996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317191 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317191 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317159 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnph6\" (UniqueName: \"kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317191 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317337 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317337 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317730 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.317965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.317944 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.318205 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.318175 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.318654 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.318631 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.318880 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.318861 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:54.319721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.319706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.319891 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.319873 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.332981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.332960 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnph6\" (UniqueName: \"kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6\") pod \"console-bdd488488-frz5n\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.382607 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.382585 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:03:54.392329 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.392303 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:54.514071 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:54.513920 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:03:54.516463 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:54.516438 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677ad03d_e8dc_475e_bf8a_7ace0a420ffc.slice/crio-cceb98b93a5ab346bce03f562363a1e5e6ae1832b74e503fe2aaba22c44aaf49 WatchSource:0}: Error finding container cceb98b93a5ab346bce03f562363a1e5e6ae1832b74e503fe2aaba22c44aaf49: Status 404 returned error can't find the container with id cceb98b93a5ab346bce03f562363a1e5e6ae1832b74e503fe2aaba22c44aaf49 Jul 09 07:03:55.231569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.231526 2573 patch_prober.go:28] interesting pod/image-registry-6d67c8b49f-grbrn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 07:03:55.231949 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.231591 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" podUID="0e28daf2-4c8b-463f-862a-9c72ee91a807" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 07:03:55.304847 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.304812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:55.315450 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.315421 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wpmm" event={"ID":"e8f383b6-8045-45f4-a2a3-a50e0674607c","Type":"ContainerStarted","Data":"fda69b166e3cfa87c33a7f3a0e9d587255832ac94ed1c8972fd455e5ebf13af2"} Jul 09 07:03:55.315608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.315503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9wpmm" event={"ID":"e8f383b6-8045-45f4-a2a3-a50e0674607c","Type":"ContainerStarted","Data":"cf26e587b228cd0ffad379811dd54f94a0fbab0a28e3232578c253528247630a"} Jul 09 07:03:55.316640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.316617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdd488488-frz5n" event={"ID":"677ad03d-e8dc-475e-bf8a-7ace0a420ffc","Type":"ContainerStarted","Data":"cceb98b93a5ab346bce03f562363a1e5e6ae1832b74e503fe2aaba22c44aaf49"} Jul 09 07:03:55.333448 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.333410 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9wpmm" podStartSLOduration=2.151313886 podStartE2EDuration="3.333382944s" totalImagesPullingTime="1.115576132s" totalInitContainerRuntime="66.492926ms" isStatefulPod=true podCreationTimestamp="2026-07-09 07:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:03:55.331629034 +0000 UTC m=+90.005485885" watchObservedRunningTime="2026-07-09 07:03:55.333382944 +0000 UTC m=+90.007239790" Jul 09 07:03:55.405333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.405297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:55.507168 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.507084 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:55.780884 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.780814 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:55.923607 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:55.923580 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:56.024797 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.024767 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:56.221695 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.221669 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:03:56.344414 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.344373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:56.445681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.445652 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:56.546500 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.546426 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:56.698613 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.698573 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5df8f7c654-lb5kq"] Jul 09 07:03:56.701932 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.701908 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.704261 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.704223 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 07:03:56.705490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.705221 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 07:03:56.705490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.705289 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 07:03:56.705490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.705311 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-30aaav4h95hak\"" Jul 09 07:03:56.705490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.705287 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 07:03:56.705490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.705221 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7kxcg\"" Jul 09 07:03:56.709571 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.709409 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5df8f7c654-lb5kq"] Jul 09 07:03:56.837516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-tls\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837478 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-client-certs\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-audit-log\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-client-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nbt\" (UniqueName: \"kubernetes.io/projected/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-kube-api-access-42nbt\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.837755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.837705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-metrics-server-audit-profiles\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.938979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.938945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nbt\" (UniqueName: \"kubernetes.io/projected/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-kube-api-access-42nbt\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-metrics-server-audit-profiles\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-tls\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-client-certs\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-audit-log\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939437 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939322 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-client-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939681 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-audit-log\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.939870 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.939841 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.940256 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.940233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-metrics-server-audit-profiles\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.942180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.942154 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-client-ca-bundle\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.942275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.942254 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-client-certs\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.942354 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.942333 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-secret-metrics-server-tls\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:56.946744 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:56.946722 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nbt\" (UniqueName: \"kubernetes.io/projected/7fc9cc6f-fe90-47c9-bab3-379d15d059f3-kube-api-access-42nbt\") pod \"metrics-server-5df8f7c654-lb5kq\" (UID: \"7fc9cc6f-fe90-47c9-bab3-379d15d059f3\") " pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:57.014350 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.014312 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:03:57.155434 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.155378 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5df8f7c654-lb5kq"] Jul 09 07:03:57.158180 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:03:57.158148 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc9cc6f_fe90_47c9_bab3_379d15d059f3.slice/crio-5f7ae491c1ec962e9856abb320d1fce146163e14bebbc41a5908f353f25d8d4b WatchSource:0}: Error finding container 5f7ae491c1ec962e9856abb320d1fce146163e14bebbc41a5908f353f25d8d4b: Status 404 returned error can't find the container with id 5f7ae491c1ec962e9856abb320d1fce146163e14bebbc41a5908f353f25d8d4b Jul 09 07:03:57.168814 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.168787 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:57.233175 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.233145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:57.327568 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.327535 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" event={"ID":"7fc9cc6f-fe90-47c9-bab3-379d15d059f3","Type":"ContainerStarted","Data":"5f7ae491c1ec962e9856abb320d1fce146163e14bebbc41a5908f353f25d8d4b"} Jul 09 07:03:57.859513 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:57.859474 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:58.168614 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.168581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:58.233305 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.233274 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d67c8b49f-grbrn" Jul 09 07:03:58.347842 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.347804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:58.453177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.453054 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:58.548363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.548325 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:03:58.632435 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:03:58.632384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:00.341940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:00.341798 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdd488488-frz5n" event={"ID":"677ad03d-e8dc-475e-bf8a-7ace0a420ffc","Type":"ContainerStarted","Data":"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3"} Jul 09 07:04:00.344297 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:00.344265 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" event={"ID":"7fc9cc6f-fe90-47c9-bab3-379d15d059f3","Type":"ContainerStarted","Data":"b8b4048e8f40a00dc6e123b96e08d7d71178ec495e15011200a8fcd31ee10b7d"} Jul 09 07:04:00.358343 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:00.358286 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bdd488488-frz5n" podStartSLOduration=0.828784612 podStartE2EDuration="6.358266771s" totalImagesPullingTime="5.529482159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:04:00.357494856 +0000 UTC m=+95.031351718" watchObservedRunningTime="2026-07-09 07:04:00.358266771 +0000 UTC m=+95.032123619" Jul 09 07:04:00.373492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:00.373445 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" podStartSLOduration=1.486060396 podStartE2EDuration="4.3734298s" totalImagesPullingTime="2.887369404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:04:00.372821652 +0000 UTC m=+95.046678547" watchObservedRunningTime="2026-07-09 07:04:00.3734298 +0000 UTC m=+95.047286647" Jul 09 07:04:01.253412 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.253328 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" podUID="006a6360-e47e-460d-8c7a-db99d295a547" containerName="registry" containerID="cri-o://863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b" gracePeriod=30 Jul 09 07:04:01.486219 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.486194 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:04:01.490523 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.490503 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:04:01.582316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582232 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582292 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582320 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf2fj\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582349 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582383 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582449 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582480 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582496 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca\") pod \"006a6360-e47e-460d-8c7a-db99d295a547\" (UID: \"006a6360-e47e-460d-8c7a-db99d295a547\") " Jul 09 07:04:01.582972 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.582946 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:01.583088 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.583034 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:01.584892 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.584845 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:04:01.585133 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.585107 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj" (OuterVolumeSpecName: "kube-api-access-hf2fj") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "kube-api-access-hf2fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:04:01.585208 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.585131 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:04:01.585208 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.585150 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:04:01.585315 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.585280 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:04:01.591216 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.591194 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "006a6360-e47e-460d-8c7a-db99d295a547" (UID: "006a6360-e47e-460d-8c7a-db99d295a547"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:04:01.683994 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.683944 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-bound-sa-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.683994 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.683990 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-installation-pull-secrets\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.683994 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684003 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/006a6360-e47e-460d-8c7a-db99d295a547-image-registry-private-configuration\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.684204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684013 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-registry-certificates\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.684204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684023 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/006a6360-e47e-460d-8c7a-db99d295a547-trusted-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.684204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684031 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/006a6360-e47e-460d-8c7a-db99d295a547-ca-trust-extracted\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.684204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684039 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-registry-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:01.684204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:01.684047 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf2fj\" (UniqueName: \"kubernetes.io/projected/006a6360-e47e-460d-8c7a-db99d295a547-kube-api-access-hf2fj\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:02.354866 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.354787 2573 generic.go:356] "Generic (PLEG): container finished" podID="006a6360-e47e-460d-8c7a-db99d295a547" containerID="863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b" exitCode=0 Jul 09 07:04:02.354866 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.354837 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" Jul 09 07:04:02.355033 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.354908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" event={"ID":"006a6360-e47e-460d-8c7a-db99d295a547","Type":"ContainerDied","Data":"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b"} Jul 09 07:04:02.355033 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.354954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-677fcdd584-vkzb9" event={"ID":"006a6360-e47e-460d-8c7a-db99d295a547","Type":"ContainerDied","Data":"4f64b73281d4461a5e4ccb012aaa0da1b597310595f5a54660cc8e1459a20714"} Jul 09 07:04:02.355033 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.354976 2573 scope.go:117] "RemoveContainer" containerID="863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b" Jul 09 07:04:02.363724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.363703 2573 scope.go:117] "RemoveContainer" containerID="863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b" Jul 09 07:04:02.364220 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:04:02.364198 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b\": container with ID starting with 863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b not found: ID does not exist" containerID="863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b" Jul 09 07:04:02.364296 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.364231 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b"} err="failed to get container status \"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b\": rpc error: code = NotFound desc = could not find container \"863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b\": container with ID starting with 863073076ff8d2b685c5c7344d7e1a9fa781d65098fef60bb2f24a35be20f22b not found: ID does not exist" Jul 09 07:04:02.370479 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.370458 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:04:02.374048 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:02.374021 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-677fcdd584-vkzb9"] Jul 09 07:04:03.034309 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.034277 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:03.111573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.111543 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:03.172558 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.172525 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:03.271678 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.271640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:03.328229 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.328164 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:03.916548 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:03.916518 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="006a6360-e47e-460d-8c7a-db99d295a547" path="/var/lib/kubelet/pods/006a6360-e47e-460d-8c7a-db99d295a547/volumes" Jul 09 07:04:04.383010 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:04.382929 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:04:06.490698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:06.490667 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:06.591230 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:06.591192 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:12.068797 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:12.068765 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:12.179807 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:12.179768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:12.281589 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:12.281557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:12.381264 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:12.381183 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:14.267289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:14.267253 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vf8f8" Jul 09 07:04:17.015166 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:17.015131 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:04:17.015565 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:17.015190 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:04:18.054692 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:18.054659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:18.154369 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:18.154337 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:27.376409 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.376333 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bdd488488-frz5n" podUID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" containerName="console" containerID="cri-o://9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3" gracePeriod=15 Jul 09 07:04:27.613281 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.613254 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:04:27.672742 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672704 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.672742 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672745 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.672965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672781 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnph6\" (UniqueName: \"kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.672965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672796 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.672965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672939 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.673111 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.672991 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.673111 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.673048 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert\") pod \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\" (UID: \"677ad03d-e8dc-475e-bf8a-7ace0a420ffc\") " Jul 09 07:04:27.673197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.673157 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:27.673296 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.673272 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:27.673341 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.673294 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config" (OuterVolumeSpecName: "console-config") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:27.673385 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.673373 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca" (OuterVolumeSpecName: "service-ca") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:04:27.675312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.675279 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6" (OuterVolumeSpecName: "kube-api-access-fnph6") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "kube-api-access-fnph6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:04:27.675312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.675277 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:04:27.675482 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.675345 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "677ad03d-e8dc-475e-bf8a-7ace0a420ffc" (UID: "677ad03d-e8dc-475e-bf8a-7ace0a420ffc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:04:27.773998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.773961 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.773998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.773998 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.774211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.774011 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnph6\" (UniqueName: \"kubernetes.io/projected/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-kube-api-access-fnph6\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.774211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.774024 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.774211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.774037 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.774211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.774048 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:27.774211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:27.774059 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677ad03d-e8dc-475e-bf8a-7ace0a420ffc-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:04:28.445644 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.445611 2573 generic.go:356] "Generic (PLEG): container finished" podID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" containerID="9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3" exitCode=0 Jul 09 07:04:28.446115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.445655 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bdd488488-frz5n" Jul 09 07:04:28.446115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.445672 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdd488488-frz5n" event={"ID":"677ad03d-e8dc-475e-bf8a-7ace0a420ffc","Type":"ContainerDied","Data":"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3"} Jul 09 07:04:28.446115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.445734 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bdd488488-frz5n" event={"ID":"677ad03d-e8dc-475e-bf8a-7ace0a420ffc","Type":"ContainerDied","Data":"cceb98b93a5ab346bce03f562363a1e5e6ae1832b74e503fe2aaba22c44aaf49"} Jul 09 07:04:28.446115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.445765 2573 scope.go:117] "RemoveContainer" containerID="9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3" Jul 09 07:04:28.453820 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.453804 2573 scope.go:117] "RemoveContainer" containerID="9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3" Jul 09 07:04:28.454057 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:04:28.454038 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3\": container with ID starting with 9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3 not found: ID does not exist" containerID="9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3" Jul 09 07:04:28.454104 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.454068 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3"} err="failed to get container status \"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3\": rpc error: code = NotFound desc = could not find container \"9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3\": container with ID starting with 9fbee3d9699a015b420688882426f7348bc4b7b743c27956571afac56a0c18b3 not found: ID does not exist" Jul 09 07:04:28.464608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.464588 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:04:28.468633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:28.468613 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bdd488488-frz5n"] Jul 09 07:04:29.916310 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:29.916278 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" path="/var/lib/kubelet/pods/677ad03d-e8dc-475e-bf8a-7ace0a420ffc/volumes" Jul 09 07:04:33.027701 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:33.027668 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:33.085648 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:33.085621 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:04:37.022569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:37.022541 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:04:37.026505 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:04:37.026471 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5df8f7c654-lb5kq" Jul 09 07:05:11.736623 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:11.736586 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:11.966041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:11.966004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:21.324136 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324049 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324284 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" containerName="console" Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324295 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" containerName="console" Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324316 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="006a6360-e47e-460d-8c7a-db99d295a547" containerName="registry" Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324322 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="006a6360-e47e-460d-8c7a-db99d295a547" containerName="registry" Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324357 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="677ad03d-e8dc-475e-bf8a-7ace0a420ffc" containerName="console" Jul 09 07:05:21.324578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.324367 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="006a6360-e47e-460d-8c7a-db99d295a547" containerName="registry" Jul 09 07:05:21.325979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.325962 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.328166 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328140 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 07:05:21.328913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328892 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9l2ts\"" Jul 09 07:05:21.329036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.329014 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 07:05:21.329102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328956 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 07:05:21.329160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328962 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 07:05:21.329160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328966 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 07:05:21.329268 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.328937 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 07:05:21.329268 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.329016 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 07:05:21.333160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.333141 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 07:05:21.336694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.336674 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:05:21.379722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.379694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:21.430666 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.430628 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:21.470702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhqp6\" (UniqueName: \"kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470708 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470855 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470855 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470931 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470931 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.470931 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.470921 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571559 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571676 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhqp6\" (UniqueName: \"kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571713 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571758 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571736 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571807 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571860 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.571916 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.571875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.572931 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.572904 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.573105 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.572934 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.573235 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.573208 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.573300 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.573208 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.575618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.575562 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.575618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.575572 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.579975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.579952 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhqp6\" (UniqueName: \"kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6\") pod \"console-779f75bdd6-z68n6\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.634872 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.634835 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:21.774421 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:21.774377 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:05:21.776767 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:05:21.776739 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90f9cc4_f7ce_4563_8cad_ed78c1eb63cc.slice/crio-6ea8cf3d16a5626aa78c19c105ebef5bc87aebbb13fe17e22347b7bb2b01335f WatchSource:0}: Error finding container 6ea8cf3d16a5626aa78c19c105ebef5bc87aebbb13fe17e22347b7bb2b01335f: Status 404 returned error can't find the container with id 6ea8cf3d16a5626aa78c19c105ebef5bc87aebbb13fe17e22347b7bb2b01335f Jul 09 07:05:22.625877 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:22.625845 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779f75bdd6-z68n6" event={"ID":"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc","Type":"ContainerStarted","Data":"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43"} Jul 09 07:05:22.626250 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:22.625908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779f75bdd6-z68n6" event={"ID":"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc","Type":"ContainerStarted","Data":"6ea8cf3d16a5626aa78c19c105ebef5bc87aebbb13fe17e22347b7bb2b01335f"} Jul 09 07:05:22.643209 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:22.643165 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-779f75bdd6-z68n6" podStartSLOduration=1.6431507920000001 podStartE2EDuration="1.643150792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:05:22.642550411 +0000 UTC m=+177.316407282" watchObservedRunningTime="2026-07-09 07:05:22.643150792 +0000 UTC m=+177.317007638" Jul 09 07:05:31.635255 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.635205 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:31.635255 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.635272 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:31.639978 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.639958 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:31.660409 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.660366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:05:31.830123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.830083 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:31.930357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:31.930264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:05:32.002291 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:05:32.002249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:12.923771 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:12.923736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:12.975682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:12.975646 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:26.985363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:26.985333 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:27.039006 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:27.038971 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:37.979611 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:37.979578 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:06:37.981541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:37.981526 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:37.998039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:37.998016 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:06:38.027969 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.027938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.027969 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.027968 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.028161 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.028005 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.032334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.028057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.032334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.028495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.032334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.028569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brpz2\" (UniqueName: \"kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.032334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.028666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.070909 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.070881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:38.129236 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129199 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129236 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpz2\" (UniqueName: \"kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.129526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.129378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.130079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.130051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.130170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.130060 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.130170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.130116 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.130475 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.130454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.131801 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.131779 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.131887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.131808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.136429 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.136410 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brpz2\" (UniqueName: \"kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2\") pod \"console-c645b97d7-7pvkr\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.290686 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.290593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:38.421474 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.421312 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:06:38.423981 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:06:38.423946 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ca8c21e_2e96_4dc0_a9ce_48b62f84be55.slice/crio-4f34d2539efec2f830c3bdce20d151a45d8ca7604fdbae28a7202710ce616a42 WatchSource:0}: Error finding container 4f34d2539efec2f830c3bdce20d151a45d8ca7604fdbae28a7202710ce616a42: Status 404 returned error can't find the container with id 4f34d2539efec2f830c3bdce20d151a45d8ca7604fdbae28a7202710ce616a42 Jul 09 07:06:38.883001 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.882968 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c645b97d7-7pvkr" event={"ID":"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55","Type":"ContainerStarted","Data":"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463"} Jul 09 07:06:38.883161 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.883016 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c645b97d7-7pvkr" event={"ID":"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55","Type":"ContainerStarted","Data":"4f34d2539efec2f830c3bdce20d151a45d8ca7604fdbae28a7202710ce616a42"} Jul 09 07:06:38.900265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.900214 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c645b97d7-7pvkr" podStartSLOduration=1.900200008 podStartE2EDuration="1.900200008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:06:38.898739052 +0000 UTC m=+253.572595917" watchObservedRunningTime="2026-07-09 07:06:38.900200008 +0000 UTC m=+253.574056855" Jul 09 07:06:38.950568 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:38.950520 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:39.012472 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:39.012443 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:48.038554 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.038438 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:48.137547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.137517 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:48.290822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.290737 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:48.290822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.290801 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:48.295543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.295519 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:48.919321 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.919291 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:06:48.962412 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:48.962368 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:06:49.065799 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:49.065772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:49.121686 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:49.121657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:06:49.174813 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:06:49.174736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:07:13.982289 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:13.982235 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-779f75bdd6-z68n6" podUID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" containerName="console" containerID="cri-o://79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43" gracePeriod=15 Jul 09 07:07:14.232794 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.232734 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:07:14.301270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301238 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301274 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301499 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301297 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301499 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301336 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhqp6\" (UniqueName: \"kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301499 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301377 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301499 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301425 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301499 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301456 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config\") pod \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\" (UID: \"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc\") " Jul 09 07:07:14.301811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301775 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:07:14.301944 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301821 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config" (OuterVolumeSpecName: "console-config") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:07:14.301944 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.301842 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:07:14.302103 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.302072 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca" (OuterVolumeSpecName: "service-ca") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:07:14.303592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.303569 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:07:14.303698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.303594 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6" (OuterVolumeSpecName: "kube-api-access-vhqp6") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "kube-api-access-vhqp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:07:14.303698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.303679 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" (UID: "f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:07:14.402152 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402116 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402152 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402142 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402152 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402152 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402152 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402162 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402418 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402170 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402418 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402179 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhqp6\" (UniqueName: \"kubernetes.io/projected/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-kube-api-access-vhqp6\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:14.402418 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:14.402187 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:07:15.002277 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.002247 2573 generic.go:356] "Generic (PLEG): container finished" podID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" containerID="79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43" exitCode=0 Jul 09 07:07:15.002703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.002290 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779f75bdd6-z68n6" Jul 09 07:07:15.002703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.002321 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779f75bdd6-z68n6" event={"ID":"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc","Type":"ContainerDied","Data":"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43"} Jul 09 07:07:15.002703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.002356 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779f75bdd6-z68n6" event={"ID":"f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc","Type":"ContainerDied","Data":"6ea8cf3d16a5626aa78c19c105ebef5bc87aebbb13fe17e22347b7bb2b01335f"} Jul 09 07:07:15.002703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.002377 2573 scope.go:117] "RemoveContainer" containerID="79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43" Jul 09 07:07:15.011112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.011093 2573 scope.go:117] "RemoveContainer" containerID="79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43" Jul 09 07:07:15.011362 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:07:15.011344 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43\": container with ID starting with 79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43 not found: ID does not exist" containerID="79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43" Jul 09 07:07:15.011448 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.011370 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43"} err="failed to get container status \"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43\": rpc error: code = NotFound desc = could not find container \"79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43\": container with ID starting with 79f034ae950364945f753065aa7e01243982767ef01d2235a1714d7c65613d43 not found: ID does not exist" Jul 09 07:07:15.021939 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.021919 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:07:15.028371 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.028350 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-779f75bdd6-z68n6"] Jul 09 07:07:15.916665 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:15.916628 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" path="/var/lib/kubelet/pods/f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc/volumes" Jul 09 07:07:17.626539 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:17.626509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:07:25.721288 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:25.721254 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:07:25.722401 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:25.722363 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:07:25.728500 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:25.728466 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 07:07:47.030177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.030146 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:07:47.643849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.643815 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj"] Jul 09 07:07:47.644069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.644057 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" containerName="console" Jul 09 07:07:47.644117 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.644072 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" containerName="console" Jul 09 07:07:47.644150 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.644118 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f90f9cc4-f7ce-4563-8cad-ed78c1eb63cc" containerName="console" Jul 09 07:07:47.645997 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.645981 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.647997 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.647974 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 07:07:47.648648 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.648630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kk85f\"" Jul 09 07:07:47.648748 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.648629 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 07:07:47.657881 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.657836 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj"] Jul 09 07:07:47.726187 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.726151 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t578\" (UniqueName: \"kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.726368 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.726208 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.726368 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.726231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.826899 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.826866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t578\" (UniqueName: \"kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.827108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.826925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.827108 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.826955 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.827359 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.827337 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.827460 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.827364 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.835928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.835906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t578\" (UniqueName: \"kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:47.954940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:47.954858 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:07:48.077671 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:48.077649 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj"] Jul 09 07:07:48.080113 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:07:48.080076 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07679e8a_50ab_406d_bd86_cd42a43192e6.slice/crio-84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511 WatchSource:0}: Error finding container 84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511: Status 404 returned error can't find the container with id 84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511 Jul 09 07:07:48.082461 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:48.082441 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:07:48.113102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:48.113067 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" event={"ID":"07679e8a-50ab-406d-bd86-cd42a43192e6","Type":"ContainerStarted","Data":"84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511"} Jul 09 07:07:49.176082 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:49.176029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:07:59.156358 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:59.156325 2573 generic.go:356] "Generic (PLEG): container finished" podID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerID="3a751e72c62b56c0aba8c2a6539d86bed3c6be466d7ae0853a1c55fa2363bba2" exitCode=0 Jul 09 07:07:59.156792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:07:59.156422 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" event={"ID":"07679e8a-50ab-406d-bd86-cd42a43192e6","Type":"ContainerDied","Data":"3a751e72c62b56c0aba8c2a6539d86bed3c6be466d7ae0853a1c55fa2363bba2"} Jul 09 07:08:02.169541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:02.169504 2573 generic.go:356] "Generic (PLEG): container finished" podID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerID="7b2fc1164697fbf553786b2eb02d0ff8da52d084160c2e0cf4c46c9d20b7c267" exitCode=0 Jul 09 07:08:02.170067 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:02.169557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" event={"ID":"07679e8a-50ab-406d-bd86-cd42a43192e6","Type":"ContainerDied","Data":"7b2fc1164697fbf553786b2eb02d0ff8da52d084160c2e0cf4c46c9d20b7c267"} Jul 09 07:08:11.207167 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:11.207132 2573 generic.go:356] "Generic (PLEG): container finished" podID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerID="3bf67bb372c803d968228c6507103331ac3eed0fe12840f00625cacab30db705" exitCode=0 Jul 09 07:08:11.207655 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:11.207175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" event={"ID":"07679e8a-50ab-406d-bd86-cd42a43192e6","Type":"ContainerDied","Data":"3bf67bb372c803d968228c6507103331ac3eed0fe12840f00625cacab30db705"} Jul 09 07:08:12.381976 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.381941 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:08:12.528423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.528324 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle\") pod \"07679e8a-50ab-406d-bd86-cd42a43192e6\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " Jul 09 07:08:12.528423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.528367 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t578\" (UniqueName: \"kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578\") pod \"07679e8a-50ab-406d-bd86-cd42a43192e6\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " Jul 09 07:08:12.528423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.528423 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util\") pod \"07679e8a-50ab-406d-bd86-cd42a43192e6\" (UID: \"07679e8a-50ab-406d-bd86-cd42a43192e6\") " Jul 09 07:08:12.529060 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.529034 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle" (OuterVolumeSpecName: "bundle") pod "07679e8a-50ab-406d-bd86-cd42a43192e6" (UID: "07679e8a-50ab-406d-bd86-cd42a43192e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:08:12.530701 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.530670 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578" (OuterVolumeSpecName: "kube-api-access-2t578") pod "07679e8a-50ab-406d-bd86-cd42a43192e6" (UID: "07679e8a-50ab-406d-bd86-cd42a43192e6"). InnerVolumeSpecName "kube-api-access-2t578". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:08:12.533570 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.533548 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util" (OuterVolumeSpecName: "util") pod "07679e8a-50ab-406d-bd86-cd42a43192e6" (UID: "07679e8a-50ab-406d-bd86-cd42a43192e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:08:12.629938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.629896 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:12.629938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.629928 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07679e8a-50ab-406d-bd86-cd42a43192e6-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:12.629938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:12.629937 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t578\" (UniqueName: \"kubernetes.io/projected/07679e8a-50ab-406d-bd86-cd42a43192e6-kube-api-access-2t578\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:13.214915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:13.214881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" event={"ID":"07679e8a-50ab-406d-bd86-cd42a43192e6","Type":"ContainerDied","Data":"84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511"} Jul 09 07:08:13.215084 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:13.214928 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84562c6a2819ce286866a9e3d91b332e220d677e8a1a216dc821c2ae12c93511" Jul 09 07:08:13.215084 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:13.214929 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vn2sj" Jul 09 07:08:18.704431 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:18.704343 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:19.553634 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:19.553600 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:20.656463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656430 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn"] Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656683 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="pull" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656694 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="pull" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656703 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="extract" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656708 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="extract" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656718 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="util" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656724 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="util" Jul 09 07:08:20.656821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.656766 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="07679e8a-50ab-406d-bd86-cd42a43192e6" containerName="extract" Jul 09 07:08:20.658485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.658458 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.660503 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.660481 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rmgxb\"" Jul 09 07:08:20.660635 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.660521 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 07:08:20.660690 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.660631 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 07:08:20.672461 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.672443 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn"] Jul 09 07:08:20.793487 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.793446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.793655 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.793494 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7szs\" (UniqueName: \"kubernetes.io/projected/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-kube-api-access-p7szs\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.893987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.893950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.893987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.893991 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7szs\" (UniqueName: \"kubernetes.io/projected/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-kube-api-access-p7szs\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.894331 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.894310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.902567 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.902541 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7szs\" (UniqueName: \"kubernetes.io/projected/e766fd4b-562a-49e2-bedc-9f2f74cf16bb-kube-api-access-p7szs\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jshn\" (UID: \"e766fd4b-562a-49e2-bedc-9f2f74cf16bb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:20.966702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:20.966621 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" Jul 09 07:08:21.112983 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:21.112936 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn"] Jul 09 07:08:21.115267 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:08:21.115227 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode766fd4b_562a_49e2_bedc_9f2f74cf16bb.slice/crio-2e0cd7bf0c9c4cbf1101b5280e3c1736865bba173fd1ba72050a5142afdc3e22 WatchSource:0}: Error finding container 2e0cd7bf0c9c4cbf1101b5280e3c1736865bba173fd1ba72050a5142afdc3e22: Status 404 returned error can't find the container with id 2e0cd7bf0c9c4cbf1101b5280e3c1736865bba173fd1ba72050a5142afdc3e22 Jul 09 07:08:21.243194 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:21.243110 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" event={"ID":"e766fd4b-562a-49e2-bedc-9f2f74cf16bb","Type":"ContainerStarted","Data":"2e0cd7bf0c9c4cbf1101b5280e3c1736865bba173fd1ba72050a5142afdc3e22"} Jul 09 07:08:22.307528 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:22.307497 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:22.408064 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:22.408026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:23.253494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:23.253457 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" event={"ID":"e766fd4b-562a-49e2-bedc-9f2f74cf16bb","Type":"ContainerStarted","Data":"2155d9fa820cf3c943c507914426646ca902fd221c4040f60f8db8df1d6214f3"} Jul 09 07:08:23.274609 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:23.274561 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jshn" podStartSLOduration=1.807311414 podStartE2EDuration="3.274546003s" totalImagesPullingTime="1.467234589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:08:23.273843323 +0000 UTC m=+357.947700192" watchObservedRunningTime="2026-07-09 07:08:23.274546003 +0000 UTC m=+357.948402849" Jul 09 07:08:23.665944 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:23.665904 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:23.780741 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:23.780695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:24.220042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.219993 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:24.349069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.349032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:24.395548 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.395518 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp"] Jul 09 07:08:24.398770 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.398746 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.401057 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.401033 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 07:08:24.401167 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.401116 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 07:08:24.401813 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.401792 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kk85f\"" Jul 09 07:08:24.407651 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.407626 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp"] Jul 09 07:08:24.526316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.526227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.526316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.526272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.526573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.526448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kbqj\" (UniqueName: \"kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.527531 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.527512 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hr7r9"] Jul 09 07:08:24.530530 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.530511 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.532540 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.532516 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4p99k\"" Jul 09 07:08:24.532540 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.532530 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 07:08:24.532669 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.532522 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 07:08:24.538772 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.538752 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hr7r9"] Jul 09 07:08:24.627590 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.627590 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fkq9\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-kube-api-access-5fkq9\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.627847 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.627847 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.627847 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kbqj\" (UniqueName: \"kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.627987 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.627926 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.628041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.628017 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.637137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.637105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kbqj\" (UniqueName: \"kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.708407 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.708368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:24.728682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.728655 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fkq9\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-kube-api-access-5fkq9\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.728792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.728768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.739754 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.739725 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.739888 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.739855 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fkq9\" (UniqueName: \"kubernetes.io/projected/07c8ef53-16e2-4833-9e17-5c95c7b76d73-kube-api-access-5fkq9\") pod \"cert-manager-webhook-6486ffd997-hr7r9\" (UID: \"07c8ef53-16e2-4833-9e17-5c95c7b76d73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.836329 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.836295 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp"] Jul 09 07:08:24.839762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.839738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:24.840332 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:08:24.840311 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137b82fe_f117_4ccd_aa11_7be05647a4c0.slice/crio-bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c WatchSource:0}: Error finding container bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c: Status 404 returned error can't find the container with id bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c Jul 09 07:08:24.973792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:24.973770 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hr7r9"] Jul 09 07:08:24.975871 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:08:24.975846 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07c8ef53_16e2_4833_9e17_5c95c7b76d73.slice/crio-6d1218fcf5ae2c137adeb73afd3fd3ff99302861dabeadc60a85e87c66a75159 WatchSource:0}: Error finding container 6d1218fcf5ae2c137adeb73afd3fd3ff99302861dabeadc60a85e87c66a75159: Status 404 returned error can't find the container with id 6d1218fcf5ae2c137adeb73afd3fd3ff99302861dabeadc60a85e87c66a75159 Jul 09 07:08:25.267640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.267607 2573 generic.go:356] "Generic (PLEG): container finished" podID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerID="37c46225ad43c99067a927972e5114adf776e37fba743001c555e26847f23545" exitCode=0 Jul 09 07:08:25.267829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.267693 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" event={"ID":"137b82fe-f117-4ccd-aa11-7be05647a4c0","Type":"ContainerDied","Data":"37c46225ad43c99067a927972e5114adf776e37fba743001c555e26847f23545"} Jul 09 07:08:25.267829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.267781 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" event={"ID":"137b82fe-f117-4ccd-aa11-7be05647a4c0","Type":"ContainerStarted","Data":"bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c"} Jul 09 07:08:25.268823 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.268792 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" event={"ID":"07c8ef53-16e2-4833-9e17-5c95c7b76d73","Type":"ContainerStarted","Data":"6d1218fcf5ae2c137adeb73afd3fd3ff99302861dabeadc60a85e87c66a75159"} Jul 09 07:08:25.828349 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.828315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:25.867687 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.866986 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26472"] Jul 09 07:08:25.870752 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.870727 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:25.873382 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.873361 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hqxdw\"" Jul 09 07:08:25.878435 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:25.878416 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26472"] Jul 09 07:08:26.039039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.038998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.039211 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.039168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzrv9\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-kube-api-access-zzrv9\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.140371 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.140335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzrv9\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-kube-api-access-zzrv9\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.140569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.140425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.148696 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.148669 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.148924 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.148906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzrv9\" (UniqueName: \"kubernetes.io/projected/695ffeac-299c-4f93-af74-71751e0210f5-kube-api-access-zzrv9\") pod \"cert-manager-cainjector-559cdd8c87-26472\" (UID: \"695ffeac-299c-4f93-af74-71751e0210f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.184737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.184706 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" Jul 09 07:08:26.330896 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.330713 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-26472"] Jul 09 07:08:26.333596 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:08:26.333571 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod695ffeac_299c_4f93_af74_71751e0210f5.slice/crio-7d81e89e84a179ce95acc7c9706d0c7defc5270f4bb60e3b8e71993da258490b WatchSource:0}: Error finding container 7d81e89e84a179ce95acc7c9706d0c7defc5270f4bb60e3b8e71993da258490b: Status 404 returned error can't find the container with id 7d81e89e84a179ce95acc7c9706d0c7defc5270f4bb60e3b8e71993da258490b Jul 09 07:08:26.617670 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:26.617577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:27.146498 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:08:27.146442 2573 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: writing blob: storing blob to file \"/var/tmp/container_images_storage411662074/1\": happened during read: unexpected EOF (while reconnecting: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/8e/8ed306a9eb2cdc7b5db57788b7a70b8d6dae2ecfc602b7662ac4cebb197a1841?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260709%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260709T070826Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=80df3cea007a5f98dd45fa72342716908f1369ccf0aa647fa8601b5e07706662®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=leader-worker-set----lws-operator-bundle&akamai_signature=exp=1783581806~hmac=806feeec84595342f93664b6fac7904586b9583d4685f3e628504effc605f279\": EOF); artifact err: provided artifact is a container image" image="registry.redhat.io/leader-worker-set/lws-operator-bundle@sha256:f641cf3b2d6659e26eb0ae49c67a3416152a2fbc2f26f490889a2bd169b9ea06" Jul 09 07:08:27.146960 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:08:27.146591 2573 kuberuntime_manager.go:1459] "Unhandled Error" err="init container pull start failed in pod 77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp_openshift-marketplace(137b82fe-f117-4ccd-aa11-7be05647a4c0): ErrImagePull: unable to pull image or OCI artifact: pull image err: writing blob: storing blob to file \"/var/tmp/container_images_storage411662074/1\": happened during read: unexpected EOF (while reconnecting: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/8e/8ed306a9eb2cdc7b5db57788b7a70b8d6dae2ecfc602b7662ac4cebb197a1841?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260709%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260709T070826Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=80df3cea007a5f98dd45fa72342716908f1369ccf0aa647fa8601b5e07706662®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=leader-worker-set----lws-operator-bundle&akamai_signature=exp=1783581806~hmac=806feeec84595342f93664b6fac7904586b9583d4685f3e628504effc605f279\": EOF); artifact err: provided artifact is a container image" logger="UnhandledError" Jul 09 07:08:27.146960 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:08:27.146655 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"pull\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: writing blob: storing blob to file \\\"/var/tmp/container_images_storage411662074/1\\\": happened during read: unexpected EOF (while reconnecting: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/8e/8ed306a9eb2cdc7b5db57788b7a70b8d6dae2ecfc602b7662ac4cebb197a1841?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260709%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260709T070826Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=80df3cea007a5f98dd45fa72342716908f1369ccf0aa647fa8601b5e07706662®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=leader-worker-set----lws-operator-bundle&akamai_signature=exp=1783581806~hmac=806feeec84595342f93664b6fac7904586b9583d4685f3e628504effc605f279\\\": EOF); artifact err: provided artifact is a container image\"" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" Jul 09 07:08:27.279860 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:27.279763 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" event={"ID":"695ffeac-299c-4f93-af74-71751e0210f5","Type":"ContainerStarted","Data":"7d81e89e84a179ce95acc7c9706d0c7defc5270f4bb60e3b8e71993da258490b"} Jul 09 07:08:27.281272 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:08:27.281218 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"pull\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/leader-worker-set/lws-operator-bundle@sha256:f641cf3b2d6659e26eb0ae49c67a3416152a2fbc2f26f490889a2bd169b9ea06\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: writing blob: storing blob to file \\\"/var/tmp/container_images_storage411662074/1\\\": happened during read: unexpected EOF (while reconnecting: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/8e/8ed306a9eb2cdc7b5db57788b7a70b8d6dae2ecfc602b7662ac4cebb197a1841?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260709%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260709T070826Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=80df3cea007a5f98dd45fa72342716908f1369ccf0aa647fa8601b5e07706662®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=leader-worker-set----lws-operator-bundle&akamai_signature=exp=1783581806~hmac=806feeec84595342f93664b6fac7904586b9583d4685f3e628504effc605f279\\\": EOF); artifact err: provided artifact is a container image\"" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" Jul 09 07:08:28.285150 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:28.285119 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" event={"ID":"07c8ef53-16e2-4833-9e17-5c95c7b76d73","Type":"ContainerStarted","Data":"9c0127c3e558d244ae26374505ca34e57a00b776a14684449cea4985332bcb29"} Jul 09 07:08:28.285603 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:28.285309 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:28.286606 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:28.286587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" event={"ID":"695ffeac-299c-4f93-af74-71751e0210f5","Type":"ContainerStarted","Data":"3aac938f07d704c164513c7fd182fffb5dc59d64bff920c5d6057d7063c006e7"} Jul 09 07:08:28.301944 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:28.301894 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" podStartSLOduration=1.744147777 podStartE2EDuration="4.301880745s" totalImagesPullingTime="2.557732968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:08:28.300184295 +0000 UTC m=+362.974041154" watchObservedRunningTime="2026-07-09 07:08:28.301880745 +0000 UTC m=+362.975737591" Jul 09 07:08:28.314314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:28.314271 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-26472" podStartSLOduration=2.114048837 podStartE2EDuration="3.31425548s" totalImagesPullingTime="1.200206643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:08:28.312944129 +0000 UTC m=+362.986800977" watchObservedRunningTime="2026-07-09 07:08:28.31425548 +0000 UTC m=+362.988112328" Jul 09 07:08:29.097070 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:29.097030 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:29.150879 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:29.150848 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:34.292592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:34.292558 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hr7r9" Jul 09 07:08:35.422939 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:35.422910 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:44.352183 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:44.352151 2573 generic.go:356] "Generic (PLEG): container finished" podID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerID="57b4780754edc814e833cfa5c3267952d8fdb8248998e8b3da73bfb544188b7f" exitCode=0 Jul 09 07:08:44.352682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:44.352209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" event={"ID":"137b82fe-f117-4ccd-aa11-7be05647a4c0","Type":"ContainerDied","Data":"57b4780754edc814e833cfa5c3267952d8fdb8248998e8b3da73bfb544188b7f"} Jul 09 07:08:45.358153 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:45.358107 2573 generic.go:356] "Generic (PLEG): container finished" podID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerID="5f38d8b6071d60e8f708543c6dfcf1cf2400ee37d7ab959b77ff7c7e3a621559" exitCode=0 Jul 09 07:08:45.358599 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:45.358167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" event={"ID":"137b82fe-f117-4ccd-aa11-7be05647a4c0","Type":"ContainerDied","Data":"5f38d8b6071d60e8f708543c6dfcf1cf2400ee37d7ab959b77ff7c7e3a621559"} Jul 09 07:08:46.490463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.490439 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:46.495565 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.495548 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util\") pod \"137b82fe-f117-4ccd-aa11-7be05647a4c0\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " Jul 09 07:08:46.495649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.495581 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle\") pod \"137b82fe-f117-4ccd-aa11-7be05647a4c0\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " Jul 09 07:08:46.495649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.495611 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kbqj\" (UniqueName: \"kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj\") pod \"137b82fe-f117-4ccd-aa11-7be05647a4c0\" (UID: \"137b82fe-f117-4ccd-aa11-7be05647a4c0\") " Jul 09 07:08:46.495951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.495923 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle" (OuterVolumeSpecName: "bundle") pod "137b82fe-f117-4ccd-aa11-7be05647a4c0" (UID: "137b82fe-f117-4ccd-aa11-7be05647a4c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:08:46.497691 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.497671 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj" (OuterVolumeSpecName: "kube-api-access-5kbqj") pod "137b82fe-f117-4ccd-aa11-7be05647a4c0" (UID: "137b82fe-f117-4ccd-aa11-7be05647a4c0"). InnerVolumeSpecName "kube-api-access-5kbqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:08:46.501937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.501914 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util" (OuterVolumeSpecName: "util") pod "137b82fe-f117-4ccd-aa11-7be05647a4c0" (UID: "137b82fe-f117-4ccd-aa11-7be05647a4c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:08:46.596659 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.596624 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:46.596659 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.596654 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kbqj\" (UniqueName: \"kubernetes.io/projected/137b82fe-f117-4ccd-aa11-7be05647a4c0-kube-api-access-5kbqj\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:46.596659 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:46.596664 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/137b82fe-f117-4ccd-aa11-7be05647a4c0-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:08:47.368208 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:47.368172 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" event={"ID":"137b82fe-f117-4ccd-aa11-7be05647a4c0","Type":"ContainerDied","Data":"bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c"} Jul 09 07:08:47.368370 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:47.368222 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjxfwp" Jul 09 07:08:47.368370 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:47.368226 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bec13439d012d4da1b8fd202ff82dcb447632b4b7a456d1d6a58025c74b56b1c" Jul 09 07:08:51.659084 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:51.659050 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:51.760731 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:51.760704 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:52.415189 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:52.415156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:52.517127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:52.517098 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:57.694237 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:57.694200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:57.793012 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:57.792977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:58.888112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:58.888073 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:58.950411 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:58.950366 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:59.520118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.520085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:08:59.610382 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610347 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj"] Jul 09 07:08:59.610651 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610637 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="util" Jul 09 07:08:59.610698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610653 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="util" Jul 09 07:08:59.610698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610662 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="pull" Jul 09 07:08:59.610698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610669 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="pull" Jul 09 07:08:59.610698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610688 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="extract" Jul 09 07:08:59.610698 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610694 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="extract" Jul 09 07:08:59.610869 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.610748 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="137b82fe-f117-4ccd-aa11-7be05647a4c0" containerName="extract" Jul 09 07:08:59.612618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.612601 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.614949 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.614922 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 07:08:59.615074 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.614927 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 07:08:59.615592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.615577 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kk85f\"" Jul 09 07:08:59.623281 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.623259 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj"] Jul 09 07:08:59.688197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.688166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.688349 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.688209 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.688349 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.688245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cbnv\" (UniqueName: \"kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.789476 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.789355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.789476 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.789429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.789476 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.789473 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cbnv\" (UniqueName: \"kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.789766 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.789747 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.789818 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.789801 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.797283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.797261 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cbnv\" (UniqueName: \"kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:08:59.921494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:08:59.921466 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:09:00.064214 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:00.064191 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj"] Jul 09 07:09:00.066785 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:09:00.066760 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf138eb_b872_4626_a163_e22b5231560b.slice/crio-df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2 WatchSource:0}: Error finding container df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2: Status 404 returned error can't find the container with id df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2 Jul 09 07:09:00.421358 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:00.421328 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbf138eb-b872-4626-a163-e22b5231560b" containerID="4725ad71462b7e2c694fac3845b1e2df894adca8a89d067be39c90f97af5b566" exitCode=0 Jul 09 07:09:00.421545 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:00.421424 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" event={"ID":"cbf138eb-b872-4626-a163-e22b5231560b","Type":"ContainerDied","Data":"4725ad71462b7e2c694fac3845b1e2df894adca8a89d067be39c90f97af5b566"} Jul 09 07:09:00.421545 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:00.421474 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" event={"ID":"cbf138eb-b872-4626-a163-e22b5231560b","Type":"ContainerStarted","Data":"df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2"} Jul 09 07:09:01.427667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:01.427631 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbf138eb-b872-4626-a163-e22b5231560b" containerID="04ec117003fc83ff0a1d2c17a1bb6a8b1deac2e3c7128d11eb2bec920a4a49cc" exitCode=0 Jul 09 07:09:01.428042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:01.427710 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" event={"ID":"cbf138eb-b872-4626-a163-e22b5231560b","Type":"ContainerDied","Data":"04ec117003fc83ff0a1d2c17a1bb6a8b1deac2e3c7128d11eb2bec920a4a49cc"} Jul 09 07:09:02.433471 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:02.433424 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbf138eb-b872-4626-a163-e22b5231560b" containerID="f490b1913cd90aa00744faf53e3652297122f8e891e31b2dfaee2c82486a3000" exitCode=0 Jul 09 07:09:02.433843 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:02.433542 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" event={"ID":"cbf138eb-b872-4626-a163-e22b5231560b","Type":"ContainerDied","Data":"f490b1913cd90aa00744faf53e3652297122f8e891e31b2dfaee2c82486a3000"} Jul 09 07:09:03.568272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.568250 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:09:03.620923 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.620892 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cbnv\" (UniqueName: \"kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv\") pod \"cbf138eb-b872-4626-a163-e22b5231560b\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " Jul 09 07:09:03.621075 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.620948 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util\") pod \"cbf138eb-b872-4626-a163-e22b5231560b\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " Jul 09 07:09:03.621075 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.620983 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle\") pod \"cbf138eb-b872-4626-a163-e22b5231560b\" (UID: \"cbf138eb-b872-4626-a163-e22b5231560b\") " Jul 09 07:09:03.621721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.621692 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle" (OuterVolumeSpecName: "bundle") pod "cbf138eb-b872-4626-a163-e22b5231560b" (UID: "cbf138eb-b872-4626-a163-e22b5231560b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:09:03.623015 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.622990 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv" (OuterVolumeSpecName: "kube-api-access-2cbnv") pod "cbf138eb-b872-4626-a163-e22b5231560b" (UID: "cbf138eb-b872-4626-a163-e22b5231560b"). InnerVolumeSpecName "kube-api-access-2cbnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:09:03.628941 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.628902 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util" (OuterVolumeSpecName: "util") pod "cbf138eb-b872-4626-a163-e22b5231560b" (UID: "cbf138eb-b872-4626-a163-e22b5231560b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:09:03.721649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.721569 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:09:03.721649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.721597 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbf138eb-b872-4626-a163-e22b5231560b-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:09:03.721649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.721613 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cbnv\" (UniqueName: \"kubernetes.io/projected/cbf138eb-b872-4626-a163-e22b5231560b-kube-api-access-2cbnv\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:09:03.749116 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.749086 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:03.818283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818248 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942"] Jul 09 07:09:03.818528 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818515 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="util" Jul 09 07:09:03.818574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818529 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="util" Jul 09 07:09:03.818574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818537 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="extract" Jul 09 07:09:03.818574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818543 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="extract" Jul 09 07:09:03.818574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818553 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="pull" Jul 09 07:09:03.818574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818559 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="pull" Jul 09 07:09:03.818715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.818613 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf138eb-b872-4626-a163-e22b5231560b" containerName="extract" Jul 09 07:09:03.820434 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.820416 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.821967 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.821936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7n6d\" (UniqueName: \"kubernetes.io/projected/9b77e34b-ea45-49a3-9d37-81a27b72bc38-kube-api-access-d7n6d\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.822065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.821974 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-metrics-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.822065 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822042 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.822151 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822064 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:03.822151 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822108 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b77e34b-ea45-49a3-9d37-81a27b72bc38-manager-config\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.822762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822742 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 07:09:03.822842 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822744 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8z8ht\"" Jul 09 07:09:03.822904 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822871 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 07:09:03.822967 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822902 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 07:09:03.822967 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.822905 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 07:09:03.823169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.823154 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 07:09:03.830655 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.830635 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942"] Jul 09 07:09:03.922573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.922543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.922715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.922584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b77e34b-ea45-49a3-9d37-81a27b72bc38-manager-config\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.922715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.922620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7n6d\" (UniqueName: \"kubernetes.io/projected/9b77e34b-ea45-49a3-9d37-81a27b72bc38-kube-api-access-d7n6d\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.922715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.922657 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-metrics-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.923304 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.923283 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9b77e34b-ea45-49a3-9d37-81a27b72bc38-manager-config\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.925086 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.925066 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.925170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.925131 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b77e34b-ea45-49a3-9d37-81a27b72bc38-metrics-cert\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:03.933581 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:03.933559 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7n6d\" (UniqueName: \"kubernetes.io/projected/9b77e34b-ea45-49a3-9d37-81a27b72bc38-kube-api-access-d7n6d\") pod \"lws-controller-manager-7bbcbb4454-2n942\" (UID: \"9b77e34b-ea45-49a3-9d37-81a27b72bc38\") " pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:04.129555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.129455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:04.255170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.255145 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942"] Jul 09 07:09:04.257313 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:09:04.257288 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b77e34b_ea45_49a3_9d37_81a27b72bc38.slice/crio-24f9dadc634dfb852d1f0213301a82f93f2a1809f96acd75433c9aaa1f626d21 WatchSource:0}: Error finding container 24f9dadc634dfb852d1f0213301a82f93f2a1809f96acd75433c9aaa1f626d21: Status 404 returned error can't find the container with id 24f9dadc634dfb852d1f0213301a82f93f2a1809f96acd75433c9aaa1f626d21 Jul 09 07:09:04.445473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.445426 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" event={"ID":"cbf138eb-b872-4626-a163-e22b5231560b","Type":"ContainerDied","Data":"df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2"} Jul 09 07:09:04.445633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.445439 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmm9nj" Jul 09 07:09:04.445633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.445501 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df329211573271b7b35c596c5fde69ad21a0de6dd057d920688c40defdd52be2" Jul 09 07:09:04.446699 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:04.446673 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" event={"ID":"9b77e34b-ea45-49a3-9d37-81a27b72bc38","Type":"ContainerStarted","Data":"24f9dadc634dfb852d1f0213301a82f93f2a1809f96acd75433c9aaa1f626d21"} Jul 09 07:09:06.457062 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:06.457022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" event={"ID":"9b77e34b-ea45-49a3-9d37-81a27b72bc38","Type":"ContainerStarted","Data":"deac6e1b1e880e03d55217cb6b48cc723a5b2996c9dd6806c2131e4f1e68fe69"} Jul 09 07:09:06.457541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:06.457166 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:06.475007 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:06.474953 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" podStartSLOduration=1.973609841 podStartE2EDuration="3.474935495s" totalImagesPullingTime="1.501325654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:09:06.474162777 +0000 UTC m=+401.148019658" watchObservedRunningTime="2026-07-09 07:09:06.474935495 +0000 UTC m=+401.148792342" Jul 09 07:09:15.446093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:15.446058 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:15.542019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:15.541973 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.027939 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.027912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.277056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.277015 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.380812 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.380723 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.431743 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.431715 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz"] Jul 09 07:09:16.433975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.433953 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.436836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.436818 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 07:09:16.436989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.436968 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 07:09:16.437064 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.437006 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 07:09:16.437269 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.437255 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l47m2\"" Jul 09 07:09:16.437438 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.437294 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 07:09:16.453563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.453539 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz"] Jul 09 07:09:16.523353 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.523323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzchf\" (UniqueName: \"kubernetes.io/projected/d22468e8-ccc7-4d4a-a46e-a77d4186320e-kube-api-access-vzchf\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.523353 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.523355 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.523555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.523382 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.523555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.523518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-tmp\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.523555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.523547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-manifests\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.560824 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.560785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.624052 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-tmp\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.624190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-manifests\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.624190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzchf\" (UniqueName: \"kubernetes.io/projected/d22468e8-ccc7-4d4a-a46e-a77d4186320e-kube-api-access-vzchf\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.624190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.624190 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.624518 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.624497 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-manifests\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.626558 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.626535 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d22468e8-ccc7-4d4a-a46e-a77d4186320e-tmp\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.626700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.626682 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.626766 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.626678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d22468e8-ccc7-4d4a-a46e-a77d4186320e-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.634497 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.634474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzchf\" (UniqueName: \"kubernetes.io/projected/d22468e8-ccc7-4d4a-a46e-a77d4186320e-kube-api-access-vzchf\") pod \"opendatahub-operator-controller-manager-b96c58b47-bz7pz\" (UID: \"d22468e8-ccc7-4d4a-a46e-a77d4186320e\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.667821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.667798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.744423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.744365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:16.759144 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.759115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.846975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.846922 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:16.881174 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.881151 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz"] Jul 09 07:09:16.883354 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:09:16.883329 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22468e8_ccc7_4d4a_a46e_a77d4186320e.slice/crio-f51121889bbeee879bc6a8d06b0f32c2b38a8bd715d040edac424d4fd777a4df WatchSource:0}: Error finding container f51121889bbeee879bc6a8d06b0f32c2b38a8bd715d040edac424d4fd777a4df: Status 404 returned error can't find the container with id f51121889bbeee879bc6a8d06b0f32c2b38a8bd715d040edac424d4fd777a4df Jul 09 07:09:16.923954 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:16.923909 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:17.464520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:17.464489 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bbcbb4454-2n942" Jul 09 07:09:17.508329 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:17.508292 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" event={"ID":"d22468e8-ccc7-4d4a-a46e-a77d4186320e","Type":"ContainerStarted","Data":"f51121889bbeee879bc6a8d06b0f32c2b38a8bd715d040edac424d4fd777a4df"} Jul 09 07:09:20.524387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:20.524300 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" event={"ID":"d22468e8-ccc7-4d4a-a46e-a77d4186320e","Type":"ContainerStarted","Data":"d79a86748381d414013068333e4360b7bd748e0f2b705b4d83c479d456191213"} Jul 09 07:09:21.530285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:21.530245 2573 generic.go:356] "Generic (PLEG): container finished" podID="d22468e8-ccc7-4d4a-a46e-a77d4186320e" containerID="d79a86748381d414013068333e4360b7bd748e0f2b705b4d83c479d456191213" exitCode=0 Jul 09 07:09:21.530732 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:21.530333 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" event={"ID":"d22468e8-ccc7-4d4a-a46e-a77d4186320e","Type":"ContainerDied","Data":"d79a86748381d414013068333e4360b7bd748e0f2b705b4d83c479d456191213"} Jul 09 07:09:22.537619 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:22.537572 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" event={"ID":"d22468e8-ccc7-4d4a-a46e-a77d4186320e","Type":"ContainerStarted","Data":"551d886bcf3f570c3276f5c483571bc0574d75250735ba6fd00816cf4c99e54f"} Jul 09 07:09:22.537992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:22.537697 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:22.559521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:22.559472 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" podStartSLOduration=1.6473524830000001 podStartE2EDuration="6.559458449s" totalImagesPullingTime="3.821146424s" totalInitContainerRuntime="1.090959542s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:09:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:09:22.557609115 +0000 UTC m=+417.231465961" watchObservedRunningTime="2026-07-09 07:09:22.559458449 +0000 UTC m=+417.233315296" Jul 09 07:09:28.434875 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:28.434837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:28.732781 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:28.732694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:33.543345 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:33.543314 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b96c58b47-bz7pz" Jul 09 07:09:34.507699 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:34.507664 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:35.625202 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:35.625149 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:35.781882 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:35.781849 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:36.502749 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:36.502715 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:36.654356 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:36.654327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:36.746781 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:36.746738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:37.039161 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:37.039129 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:37.132705 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:37.132674 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:37.244278 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:37.244240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:45.794531 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:45.794481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:45.895363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:45.895330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:46.223177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:46.223139 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:46.326716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:46.326686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:46.553408 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:46.553314 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:46.950803 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:46.950761 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:47.142762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:47.142684 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:47.252057 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:47.251980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:48.612364 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:48.612330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:48.708552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:48.708517 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:49.994162 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:49.994124 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:53.676978 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:53.676946 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:53.787037 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:53.787005 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:54.330332 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.330296 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr"] Jul 09 07:09:54.364164 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.364132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:54.406699 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.406662 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr"] Jul 09 07:09:54.406854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.406762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.409518 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.409462 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 07:09:54.409675 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.409527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 07:09:54.409675 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.409499 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 07:09:54.409789 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.409772 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pgz48\"" Jul 09 07:09:54.551894 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.551856 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.551894 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.551896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.551924 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.551958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.551989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36885451-6534-4ed7-9ebe-c95229740495-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.552018 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36885451-6534-4ed7-9ebe-c95229740495-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.552057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.552080 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.552307 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.552140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhtgw\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-kube-api-access-vhtgw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.561258 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.561225 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:54.653386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653421 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhtgw\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-kube-api-access-vhtgw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653606 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36885451-6534-4ed7-9ebe-c95229740495-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36885451-6534-4ed7-9ebe-c95229740495-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.653975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.653959 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.654079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.654062 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.654124 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.654093 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.654333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.654315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36885451-6534-4ed7-9ebe-c95229740495-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.655862 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.655843 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36885451-6534-4ed7-9ebe-c95229740495-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.656059 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.656042 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36885451-6534-4ed7-9ebe-c95229740495-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.662163 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.662142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.662314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.662291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhtgw\" (UniqueName: \"kubernetes.io/projected/36885451-6534-4ed7-9ebe-c95229740495-kube-api-access-vhtgw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf457qpr\" (UID: \"36885451-6534-4ed7-9ebe-c95229740495\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.716528 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.716498 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:54.781690 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.781659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:09:54.845915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:54.845891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr"] Jul 09 07:09:54.847574 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:09:54.847537 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36885451_6534_4ed7_9ebe_c95229740495.slice/crio-b94ae2dff5803a392e83c1e76dca82a2baa83c21ac21c79592289a940acc0ac7 WatchSource:0}: Error finding container b94ae2dff5803a392e83c1e76dca82a2baa83c21ac21c79592289a940acc0ac7: Status 404 returned error can't find the container with id b94ae2dff5803a392e83c1e76dca82a2baa83c21ac21c79592289a940acc0ac7 Jul 09 07:09:55.684339 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:55.684288 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" event={"ID":"36885451-6534-4ed7-9ebe-c95229740495","Type":"ContainerStarted","Data":"b94ae2dff5803a392e83c1e76dca82a2baa83c21ac21c79592289a940acc0ac7"} Jul 09 07:09:57.571258 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.571220 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:09:57.571534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.571294 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:09:57.571534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.571325 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:09:57.696981 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.696937 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" event={"ID":"36885451-6534-4ed7-9ebe-c95229740495","Type":"ContainerStarted","Data":"d20b4b789a0e9b0ee5d64eaf1ae6ad3556d204beaf371b39df040cdf197a451c"} Jul 09 07:09:57.716928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.716880 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:57.726300 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:57.726241 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" podStartSLOduration=1.004822143 podStartE2EDuration="3.726223449s" totalImagesPullingTime="2.721401306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:09:57.724994688 +0000 UTC m=+452.398851535" watchObservedRunningTime="2026-07-09 07:09:57.726223449 +0000 UTC m=+452.400080295" Jul 09 07:09:58.721667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:58.721625 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:59.706118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:59.706083 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:09:59.707127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:09:59.707108 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf457qpr" Jul 09 07:10:11.934794 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:11.934761 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:11.990735 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:11.990703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:22.009822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.009786 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c46fd74f7-szqcf"] Jul 09 07:10:22.012810 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.012794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.023726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.023700 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c46fd74f7-szqcf"] Jul 09 07:10:22.056308 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-trusted-ca-bundle\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056387 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbq5j\" (UniqueName: \"kubernetes.io/projected/004bbad5-f5e3-4067-b706-db4948c5faf4-kube-api-access-rbq5j\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-oauth-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056568 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056498 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-oauth-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056568 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-service-ca\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.056568 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.056543 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-console-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.069131 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.069102 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:22.124123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.124090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:22.157574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbq5j\" (UniqueName: \"kubernetes.io/projected/004bbad5-f5e3-4067-b706-db4948c5faf4-kube-api-access-rbq5j\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157703 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157604 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-oauth-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157846 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-oauth-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157880 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157868 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-service-ca\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-console-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.157973 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.157931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-trusted-ca-bundle\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.158454 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.158428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-oauth-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.158647 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.158624 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-service-ca\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.158803 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.158779 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-console-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.158841 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.158796 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004bbad5-f5e3-4067-b706-db4948c5faf4-trusted-ca-bundle\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.160247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.160225 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-oauth-config\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.160318 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.160304 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/004bbad5-f5e3-4067-b706-db4948c5faf4-console-serving-cert\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.165811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.165789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbq5j\" (UniqueName: \"kubernetes.io/projected/004bbad5-f5e3-4067-b706-db4948c5faf4-kube-api-access-rbq5j\") pod \"console-5c46fd74f7-szqcf\" (UID: \"004bbad5-f5e3-4067-b706-db4948c5faf4\") " pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.322515 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.322423 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:22.449494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.449461 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c46fd74f7-szqcf"] Jul 09 07:10:22.451274 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:22.451246 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004bbad5_f5e3_4067_b706_db4948c5faf4.slice/crio-97ea20df9d9063a10e98e5afec202967c47d9dbb8be23c231f569e998ba0576d WatchSource:0}: Error finding container 97ea20df9d9063a10e98e5afec202967c47d9dbb8be23c231f569e998ba0576d: Status 404 returned error can't find the container with id 97ea20df9d9063a10e98e5afec202967c47d9dbb8be23c231f569e998ba0576d Jul 09 07:10:22.805273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.805235 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c46fd74f7-szqcf" event={"ID":"004bbad5-f5e3-4067-b706-db4948c5faf4","Type":"ContainerStarted","Data":"302edf78a1b3dfdfb2b4f56d6042fc941ac790f3795c31a4010aae947fab5065"} Jul 09 07:10:22.805460 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.805290 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c46fd74f7-szqcf" event={"ID":"004bbad5-f5e3-4067-b706-db4948c5faf4","Type":"ContainerStarted","Data":"97ea20df9d9063a10e98e5afec202967c47d9dbb8be23c231f569e998ba0576d"} Jul 09 07:10:22.823323 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:22.823254 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c46fd74f7-szqcf" podStartSLOduration=1.823237789 podStartE2EDuration="1.823237789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:10:22.822158923 +0000 UTC m=+477.496015794" watchObservedRunningTime="2026-07-09 07:10:22.823237789 +0000 UTC m=+477.497094635" Jul 09 07:10:32.323487 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:32.323436 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:32.323978 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:32.323534 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:32.328292 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:32.328265 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:32.852455 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:32.852429 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c46fd74f7-szqcf" Jul 09 07:10:32.906757 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:32.906728 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:10:33.038148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:33.038114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:33.136179 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:33.136152 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:33.236519 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:33.236490 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:37.133653 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.133615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:37.233762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.233719 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:37.285467 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.285435 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:37.290627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.290607 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:37.293730 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.293706 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 07:10:37.294386 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.294368 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8kdlr\"" Jul 09 07:10:37.294480 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.294375 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 07:10:37.303218 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.303195 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:37.390982 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.390954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsnl5\" (UniqueName: \"kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5\") pod \"kuadrant-operator-catalog-pcpnr\" (UID: \"5aa1e7bf-8815-43d7-a6a2-bb83c796405d\") " pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:37.436750 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.436716 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:37.491712 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.491683 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsnl5\" (UniqueName: \"kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5\") pod \"kuadrant-operator-catalog-pcpnr\" (UID: \"5aa1e7bf-8815-43d7-a6a2-bb83c796405d\") " pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:37.499433 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.499405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsnl5\" (UniqueName: \"kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5\") pod \"kuadrant-operator-catalog-pcpnr\" (UID: \"5aa1e7bf-8815-43d7-a6a2-bb83c796405d\") " pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:37.599718 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.599686 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:37.657301 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.657237 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:37.725154 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.725124 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:37.729169 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:37.729141 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa1e7bf_8815_43d7_a6a2_bb83c796405d.slice/crio-92ece0c23f2d7c80221265d1d0307b6226d4f3e5c10233dea7abb7c8ad422b5e WatchSource:0}: Error finding container 92ece0c23f2d7c80221265d1d0307b6226d4f3e5c10233dea7abb7c8ad422b5e: Status 404 returned error can't find the container with id 92ece0c23f2d7c80221265d1d0307b6226d4f3e5c10233dea7abb7c8ad422b5e Jul 09 07:10:37.869826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.869790 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" event={"ID":"5aa1e7bf-8815-43d7-a6a2-bb83c796405d","Type":"ContainerStarted","Data":"92ece0c23f2d7c80221265d1d0307b6226d4f3e5c10233dea7abb7c8ad422b5e"} Jul 09 07:10:37.871200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.871173 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dkkxx"] Jul 09 07:10:37.873712 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.873692 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:37.883526 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.883505 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dkkxx"] Jul 09 07:10:37.996434 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:37.996325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb895\" (UniqueName: \"kubernetes.io/projected/1243bfae-0fd7-4ee7-8450-b48a0aacc0cf-kube-api-access-tb895\") pod \"kuadrant-operator-catalog-dkkxx\" (UID: \"1243bfae-0fd7-4ee7-8450-b48a0aacc0cf\") " pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:38.097432 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:38.097378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb895\" (UniqueName: \"kubernetes.io/projected/1243bfae-0fd7-4ee7-8450-b48a0aacc0cf-kube-api-access-tb895\") pod \"kuadrant-operator-catalog-dkkxx\" (UID: \"1243bfae-0fd7-4ee7-8450-b48a0aacc0cf\") " pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:38.106098 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:38.106072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb895\" (UniqueName: \"kubernetes.io/projected/1243bfae-0fd7-4ee7-8450-b48a0aacc0cf-kube-api-access-tb895\") pod \"kuadrant-operator-catalog-dkkxx\" (UID: \"1243bfae-0fd7-4ee7-8450-b48a0aacc0cf\") " pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:38.183107 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:38.183075 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:38.323265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:38.323240 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dkkxx"] Jul 09 07:10:38.325375 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:38.325346 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1243bfae_0fd7_4ee7_8450_b48a0aacc0cf.slice/crio-29960819ccfd3b309bc2a2c81238f73208e4ea02a003a0d9a48dd9bb5daa6d70 WatchSource:0}: Error finding container 29960819ccfd3b309bc2a2c81238f73208e4ea02a003a0d9a48dd9bb5daa6d70: Status 404 returned error can't find the container with id 29960819ccfd3b309bc2a2c81238f73208e4ea02a003a0d9a48dd9bb5daa6d70 Jul 09 07:10:38.877192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:38.877151 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" event={"ID":"1243bfae-0fd7-4ee7-8450-b48a0aacc0cf","Type":"ContainerStarted","Data":"29960819ccfd3b309bc2a2c81238f73208e4ea02a003a0d9a48dd9bb5daa6d70"} Jul 09 07:10:40.887522 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:40.887488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" event={"ID":"1243bfae-0fd7-4ee7-8450-b48a0aacc0cf","Type":"ContainerStarted","Data":"8f410fc4c9b25a3c05884d8b7bf048424a45702789cb35e22dfab8d9e0195e7a"} Jul 09 07:10:40.888812 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:40.888789 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" event={"ID":"5aa1e7bf-8815-43d7-a6a2-bb83c796405d","Type":"ContainerStarted","Data":"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5"} Jul 09 07:10:40.888911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:40.888862 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" podUID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" containerName="registry-server" containerID="cri-o://3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5" gracePeriod=2 Jul 09 07:10:40.902219 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:40.902177 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" podStartSLOduration=2.242973621 podStartE2EDuration="3.902160921s" totalImagesPullingTime="1.6591873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:10:40.901074391 +0000 UTC m=+495.574931237" watchObservedRunningTime="2026-07-09 07:10:40.902160921 +0000 UTC m=+495.576017767" Jul 09 07:10:40.915780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:40.915734 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" podStartSLOduration=1.663491565 podStartE2EDuration="3.9157166s" totalImagesPullingTime="2.252225035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:10:40.915149874 +0000 UTC m=+495.589006747" watchObservedRunningTime="2026-07-09 07:10:40.9157166 +0000 UTC m=+495.589573450" Jul 09 07:10:41.123573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.123543 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:41.226809 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.226723 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsnl5\" (UniqueName: \"kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5\") pod \"5aa1e7bf-8815-43d7-a6a2-bb83c796405d\" (UID: \"5aa1e7bf-8815-43d7-a6a2-bb83c796405d\") " Jul 09 07:10:41.228861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.228831 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5" (OuterVolumeSpecName: "kube-api-access-jsnl5") pod "5aa1e7bf-8815-43d7-a6a2-bb83c796405d" (UID: "5aa1e7bf-8815-43d7-a6a2-bb83c796405d"). InnerVolumeSpecName "kube-api-access-jsnl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:41.327547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.327512 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsnl5\" (UniqueName: \"kubernetes.io/projected/5aa1e7bf-8815-43d7-a6a2-bb83c796405d-kube-api-access-jsnl5\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:41.894260 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.894224 2573 generic.go:356] "Generic (PLEG): container finished" podID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" containerID="3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5" exitCode=0 Jul 09 07:10:41.894260 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.894262 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" Jul 09 07:10:41.894724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.894296 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" event={"ID":"5aa1e7bf-8815-43d7-a6a2-bb83c796405d","Type":"ContainerDied","Data":"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5"} Jul 09 07:10:41.894724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.894336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcpnr" event={"ID":"5aa1e7bf-8815-43d7-a6a2-bb83c796405d","Type":"ContainerDied","Data":"92ece0c23f2d7c80221265d1d0307b6226d4f3e5c10233dea7abb7c8ad422b5e"} Jul 09 07:10:41.894724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.894359 2573 scope.go:117] "RemoveContainer" containerID="3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5" Jul 09 07:10:41.909337 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.909320 2573 scope.go:117] "RemoveContainer" containerID="3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5" Jul 09 07:10:41.909614 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:10:41.909596 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5\": container with ID starting with 3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5 not found: ID does not exist" containerID="3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5" Jul 09 07:10:41.909662 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.909622 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5"} err="failed to get container status \"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5\": rpc error: code = NotFound desc = could not find container \"3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5\": container with ID starting with 3f6f0f04b212acd8157d5532d1728196a5da3ee772f97336a03a8e4ebb3853c5 not found: ID does not exist" Jul 09 07:10:41.917702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.917678 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:41.920915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:41.920895 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcpnr"] Jul 09 07:10:42.618021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:42.617990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:10:43.916851 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:43.916820 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" path="/var/lib/kubelet/pods/5aa1e7bf-8815-43d7-a6a2-bb83c796405d/volumes" Jul 09 07:10:48.183707 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:48.183670 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:48.184106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:48.183741 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:48.205985 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:48.205956 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:48.946226 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:48.946195 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dkkxx" Jul 09 07:10:52.484158 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.484121 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm"] Jul 09 07:10:52.484542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.484451 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" containerName="registry-server" Jul 09 07:10:52.484542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.484464 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" containerName="registry-server" Jul 09 07:10:52.484542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.484528 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aa1e7bf-8815-43d7-a6a2-bb83c796405d" containerName="registry-server" Jul 09 07:10:52.487186 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.487169 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.492351 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.492332 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9hnnf\"" Jul 09 07:10:52.498881 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.498842 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm"] Jul 09 07:10:52.623573 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.623507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.623747 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.623705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.623816 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.623764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgm5t\" (UniqueName: \"kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.724656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.724603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.724829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.724695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.724829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.724726 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgm5t\" (UniqueName: \"kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.725101 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.725077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.725209 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.725103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.733919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.733898 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgm5t\" (UniqueName: \"kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.796604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.796523 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:52.927566 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.927542 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm"] Jul 09 07:10:52.929718 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:52.929684 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be316c7_df7b_478b_979e_6200b7ca5255.slice/crio-d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62 WatchSource:0}: Error finding container d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62: Status 404 returned error can't find the container with id d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62 Jul 09 07:10:52.941165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:52.941139 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" event={"ID":"7be316c7-df7b-478b-979e-6200b7ca5255","Type":"ContainerStarted","Data":"d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62"} Jul 09 07:10:53.299886 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.299851 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj"] Jul 09 07:10:53.303122 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.303107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.313313 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.313287 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj"] Jul 09 07:10:53.431848 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.431804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.432027 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.431887 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.432027 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.431920 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-886br\" (UniqueName: \"kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.533270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.533225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.533732 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.533334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886br\" (UniqueName: \"kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.533732 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.533438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.533732 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.533623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.533857 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.533759 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.541892 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.541871 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-886br\" (UniqueName: \"kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.612743 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.612656 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:53.683911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.683877 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2"] Jul 09 07:10:53.688125 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.688102 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.695210 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.695186 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2"] Jul 09 07:10:53.744727 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.744703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj"] Jul 09 07:10:53.746699 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:53.746674 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1853565c_d4f4_44cb_b698_5b0a4c1dd611.slice/crio-1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6 WatchSource:0}: Error finding container 1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6: Status 404 returned error can't find the container with id 1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6 Jul 09 07:10:53.836195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.836159 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.836369 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.836242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.836369 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.836312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97ljt\" (UniqueName: \"kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.937540 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.937506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97ljt\" (UniqueName: \"kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.937738 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.937551 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.937738 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.937604 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.937997 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.937973 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.938039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.937990 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.946251 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.946226 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97ljt\" (UniqueName: \"kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:53.947729 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.947702 2573 generic.go:356] "Generic (PLEG): container finished" podID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerID="3d7078c09a98907534d764393b49b2581b086675be03ca0ca3d297c1281bc6aa" exitCode=0 Jul 09 07:10:53.947820 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.947781 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" event={"ID":"1853565c-d4f4-44cb-b698-5b0a4c1dd611","Type":"ContainerDied","Data":"3d7078c09a98907534d764393b49b2581b086675be03ca0ca3d297c1281bc6aa"} Jul 09 07:10:53.947881 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.947830 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" event={"ID":"1853565c-d4f4-44cb-b698-5b0a4c1dd611","Type":"ContainerStarted","Data":"1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6"} Jul 09 07:10:53.949581 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.949563 2573 generic.go:356] "Generic (PLEG): container finished" podID="7be316c7-df7b-478b-979e-6200b7ca5255" containerID="76a9d4bf51134e378a94d810d009035a99e0d40308c32daa04d39f79a6ad1c77" exitCode=0 Jul 09 07:10:53.949677 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:53.949602 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" event={"ID":"7be316c7-df7b-478b-979e-6200b7ca5255","Type":"ContainerDied","Data":"76a9d4bf51134e378a94d810d009035a99e0d40308c32daa04d39f79a6ad1c77"} Jul 09 07:10:54.001593 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.001556 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:54.090538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.090504 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45"] Jul 09 07:10:54.098447 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.098419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.101672 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.101639 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45"] Jul 09 07:10:54.136335 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.136313 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2"] Jul 09 07:10:54.138728 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:54.138699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c0e91a_a16a_49bd_a8e3_40e7b3baf599.slice/crio-1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5 WatchSource:0}: Error finding container 1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5: Status 404 returned error can't find the container with id 1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5 Jul 09 07:10:54.240355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.240322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.240355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.240366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdcm\" (UniqueName: \"kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.240531 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.240511 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.341755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.341716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.341755 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.341768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.341963 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.341790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdcm\" (UniqueName: \"kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.342112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.342092 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.342167 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.342146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.351118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.351096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdcm\" (UniqueName: \"kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.410270 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.410238 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:54.539930 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.539906 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45"] Jul 09 07:10:54.541275 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:10:54.541252 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb5e708_5449_4763_949a_d1da3d934776.slice/crio-a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10 WatchSource:0}: Error finding container a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10: Status 404 returned error can't find the container with id a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10 Jul 09 07:10:54.956341 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.956308 2573 generic.go:356] "Generic (PLEG): container finished" podID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerID="6962f39810e3b5526d4ab2473da70a4df0f145c81c1dddb29d02f3941b25e310" exitCode=0 Jul 09 07:10:54.956516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.956379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerDied","Data":"6962f39810e3b5526d4ab2473da70a4df0f145c81c1dddb29d02f3941b25e310"} Jul 09 07:10:54.956516 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.956468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerStarted","Data":"1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5"} Jul 09 07:10:54.957908 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.957886 2573 generic.go:356] "Generic (PLEG): container finished" podID="ddb5e708-5449-4763-949a-d1da3d934776" containerID="d8f1417399132ca729f47adba9f44ab58d6eece47ae0e776b681f5f1890db3a2" exitCode=0 Jul 09 07:10:54.958036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.957967 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" event={"ID":"ddb5e708-5449-4763-949a-d1da3d934776","Type":"ContainerDied","Data":"d8f1417399132ca729f47adba9f44ab58d6eece47ae0e776b681f5f1890db3a2"} Jul 09 07:10:54.958036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.958017 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" event={"ID":"ddb5e708-5449-4763-949a-d1da3d934776","Type":"ContainerStarted","Data":"a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10"} Jul 09 07:10:54.963181 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.963161 2573 generic.go:356] "Generic (PLEG): container finished" podID="7be316c7-df7b-478b-979e-6200b7ca5255" containerID="5a2d36419bd017550f2edad08b8d7c6d4bf9c8461fecbbfc086ed54d693da79c" exitCode=0 Jul 09 07:10:54.963271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:54.963234 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" event={"ID":"7be316c7-df7b-478b-979e-6200b7ca5255","Type":"ContainerDied","Data":"5a2d36419bd017550f2edad08b8d7c6d4bf9c8461fecbbfc086ed54d693da79c"} Jul 09 07:10:55.972253 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:55.972224 2573 generic.go:356] "Generic (PLEG): container finished" podID="7be316c7-df7b-478b-979e-6200b7ca5255" containerID="e40250485e61818e31a8a7f4d5be6ab798b6081781fe4c97a904f0d3e5e945e2" exitCode=0 Jul 09 07:10:55.972605 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:55.972306 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" event={"ID":"7be316c7-df7b-478b-979e-6200b7ca5255","Type":"ContainerDied","Data":"e40250485e61818e31a8a7f4d5be6ab798b6081781fe4c97a904f0d3e5e945e2"} Jul 09 07:10:55.974041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:55.974021 2573 generic.go:356] "Generic (PLEG): container finished" podID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerID="129fbfe20761bb2e929f9c085655d9ad483194e64a8af03973a35ed7dc800526" exitCode=0 Jul 09 07:10:55.974126 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:55.974097 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" event={"ID":"1853565c-d4f4-44cb-b698-5b0a4c1dd611","Type":"ContainerDied","Data":"129fbfe20761bb2e929f9c085655d9ad483194e64a8af03973a35ed7dc800526"} Jul 09 07:10:55.975878 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:55.975857 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerStarted","Data":"0184132e1eb74ce499a98ffa3500fba77a584e85f0d047ed24676479c8a761ea"} Jul 09 07:10:56.981899 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.981866 2573 generic.go:356] "Generic (PLEG): container finished" podID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerID="139d5e3b30de068b5010650e4a9b6aec346ebc5b4d09ceaee8d74795110faae6" exitCode=0 Jul 09 07:10:56.982291 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.981950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" event={"ID":"1853565c-d4f4-44cb-b698-5b0a4c1dd611","Type":"ContainerDied","Data":"139d5e3b30de068b5010650e4a9b6aec346ebc5b4d09ceaee8d74795110faae6"} Jul 09 07:10:56.983833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.983813 2573 generic.go:356] "Generic (PLEG): container finished" podID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerID="0184132e1eb74ce499a98ffa3500fba77a584e85f0d047ed24676479c8a761ea" exitCode=0 Jul 09 07:10:56.983833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.983829 2573 generic.go:356] "Generic (PLEG): container finished" podID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerID="52f8926e67bda737206a2e0c67f149b7d543ba48a2ae8b4ecd8b6a053473d270" exitCode=0 Jul 09 07:10:56.983988 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.983895 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerDied","Data":"0184132e1eb74ce499a98ffa3500fba77a584e85f0d047ed24676479c8a761ea"} Jul 09 07:10:56.983988 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.983932 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerDied","Data":"52f8926e67bda737206a2e0c67f149b7d543ba48a2ae8b4ecd8b6a053473d270"} Jul 09 07:10:56.985379 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.985357 2573 generic.go:356] "Generic (PLEG): container finished" podID="ddb5e708-5449-4763-949a-d1da3d934776" containerID="00679b2d82c7f951fe06b865a46f6cc90c64cb793925bcd835e9f92290edce98" exitCode=0 Jul 09 07:10:56.985500 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:56.985381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" event={"ID":"ddb5e708-5449-4763-949a-d1da3d934776","Type":"ContainerDied","Data":"00679b2d82c7f951fe06b865a46f6cc90c64cb793925bcd835e9f92290edce98"} Jul 09 07:10:57.125292 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.125269 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:57.266604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.266512 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util\") pod \"7be316c7-df7b-478b-979e-6200b7ca5255\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " Jul 09 07:10:57.266783 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.266627 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgm5t\" (UniqueName: \"kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t\") pod \"7be316c7-df7b-478b-979e-6200b7ca5255\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " Jul 09 07:10:57.266783 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.266684 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle\") pod \"7be316c7-df7b-478b-979e-6200b7ca5255\" (UID: \"7be316c7-df7b-478b-979e-6200b7ca5255\") " Jul 09 07:10:57.267244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.267215 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle" (OuterVolumeSpecName: "bundle") pod "7be316c7-df7b-478b-979e-6200b7ca5255" (UID: "7be316c7-df7b-478b-979e-6200b7ca5255"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:57.268689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.268663 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t" (OuterVolumeSpecName: "kube-api-access-mgm5t") pod "7be316c7-df7b-478b-979e-6200b7ca5255" (UID: "7be316c7-df7b-478b-979e-6200b7ca5255"). InnerVolumeSpecName "kube-api-access-mgm5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:57.271764 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.271731 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util" (OuterVolumeSpecName: "util") pod "7be316c7-df7b-478b-979e-6200b7ca5255" (UID: "7be316c7-df7b-478b-979e-6200b7ca5255"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:57.368143 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.368107 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgm5t\" (UniqueName: \"kubernetes.io/projected/7be316c7-df7b-478b-979e-6200b7ca5255-kube-api-access-mgm5t\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:57.368143 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.368136 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:57.368143 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.368145 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7be316c7-df7b-478b-979e-6200b7ca5255-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:57.928811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.928779 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c645b97d7-7pvkr" podUID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" containerName="console" containerID="cri-o://c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463" gracePeriod=15 Jul 09 07:10:57.992252 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.992191 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" Jul 09 07:10:57.992252 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.992211 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm" event={"ID":"7be316c7-df7b-478b-979e-6200b7ca5255","Type":"ContainerDied","Data":"d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62"} Jul 09 07:10:57.992642 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.992260 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d298a1f7a603a7ee86f5a002251ff204be5e5cdfd1831f72e8dabfcc60637a62" Jul 09 07:10:57.994522 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.994491 2573 generic.go:356] "Generic (PLEG): container finished" podID="ddb5e708-5449-4763-949a-d1da3d934776" containerID="8411ac2b86f64a3a603e9ef48eff85432b64f56c75603ad2e8d9fda1533cbc44" exitCode=0 Jul 09 07:10:57.994640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:57.994523 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" event={"ID":"ddb5e708-5449-4763-949a-d1da3d934776","Type":"ContainerDied","Data":"8411ac2b86f64a3a603e9ef48eff85432b64f56c75603ad2e8d9fda1533cbc44"} Jul 09 07:10:58.147990 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.147967 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:58.151032 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.151012 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:58.174617 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174588 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-886br\" (UniqueName: \"kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br\") pod \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " Jul 09 07:10:58.174749 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174624 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle\") pod \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " Jul 09 07:10:58.174749 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174654 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util\") pod \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " Jul 09 07:10:58.174749 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174711 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle\") pod \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\" (UID: \"1853565c-d4f4-44cb-b698-5b0a4c1dd611\") " Jul 09 07:10:58.174749 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174733 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util\") pod \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " Jul 09 07:10:58.175089 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.174790 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97ljt\" (UniqueName: \"kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt\") pod \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\" (UID: \"30c0e91a-a16a-49bd-a8e3-40e7b3baf599\") " Jul 09 07:10:58.175772 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.175652 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle" (OuterVolumeSpecName: "bundle") pod "30c0e91a-a16a-49bd-a8e3-40e7b3baf599" (UID: "30c0e91a-a16a-49bd-a8e3-40e7b3baf599"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:58.175885 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.175769 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle" (OuterVolumeSpecName: "bundle") pod "1853565c-d4f4-44cb-b698-5b0a4c1dd611" (UID: "1853565c-d4f4-44cb-b698-5b0a4c1dd611"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:58.177226 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.177201 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br" (OuterVolumeSpecName: "kube-api-access-886br") pod "1853565c-d4f4-44cb-b698-5b0a4c1dd611" (UID: "1853565c-d4f4-44cb-b698-5b0a4c1dd611"). InnerVolumeSpecName "kube-api-access-886br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:58.177417 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.177372 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt" (OuterVolumeSpecName: "kube-api-access-97ljt") pod "30c0e91a-a16a-49bd-a8e3-40e7b3baf599" (UID: "30c0e91a-a16a-49bd-a8e3-40e7b3baf599"). InnerVolumeSpecName "kube-api-access-97ljt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:58.181827 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.181798 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util" (OuterVolumeSpecName: "util") pod "1853565c-d4f4-44cb-b698-5b0a4c1dd611" (UID: "1853565c-d4f4-44cb-b698-5b0a4c1dd611"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:58.182053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.182029 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util" (OuterVolumeSpecName: "util") pod "30c0e91a-a16a-49bd-a8e3-40e7b3baf599" (UID: "30c0e91a-a16a-49bd-a8e3-40e7b3baf599"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:58.187786 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.187765 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:10:58.275564 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275479 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.275688 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275574 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.275688 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275628 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.275688 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275668 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.275841 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275690 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.275841 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.275758 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brpz2\" (UniqueName: \"kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.276033 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276012 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97ljt\" (UniqueName: \"kubernetes.io/projected/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-kube-api-access-97ljt\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.276106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276033 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-886br\" (UniqueName: \"kubernetes.io/projected/1853565c-d4f4-44cb-b698-5b0a4c1dd611-kube-api-access-886br\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.276106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276033 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:10:58.276106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276038 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca" (OuterVolumeSpecName: "service-ca") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:10:58.276106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276046 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.276106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276105 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.276285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276108 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:10:58.276285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276124 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1853565c-d4f4-44cb-b698-5b0a4c1dd611-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.276285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.276139 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30c0e91a-a16a-49bd-a8e3-40e7b3baf599-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.277672 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.277647 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:10:58.277765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.277752 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:10:58.277835 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.277816 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2" (OuterVolumeSpecName: "kube-api-access-brpz2") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "kube-api-access-brpz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:58.376364 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376334 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config\") pod \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\" (UID: \"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55\") " Jul 09 07:10:58.376535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376495 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376506 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376514 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376524 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376534 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brpz2\" (UniqueName: \"kubernetes.io/projected/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-kube-api-access-brpz2\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376678 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376542 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:58.376678 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.376657 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config" (OuterVolumeSpecName: "console-config") pod "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" (UID: "0ca8c21e-2e96-4dc0-a9ce-48b62f84be55"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:10:58.476814 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:58.476782 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:59.001179 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.001099 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" Jul 09 07:10:59.001609 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.001099 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj" event={"ID":"1853565c-d4f4-44cb-b698-5b0a4c1dd611","Type":"ContainerDied","Data":"1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6"} Jul 09 07:10:59.001609 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.001228 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c358900e6e29c95f5f85c072c98c46c81c65290360a46e8d6664f0d930e0cd6" Jul 09 07:10:59.002468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.002446 2573 generic.go:356] "Generic (PLEG): container finished" podID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" containerID="c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463" exitCode=0 Jul 09 07:10:59.002579 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.002502 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c645b97d7-7pvkr" Jul 09 07:10:59.002579 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.002520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c645b97d7-7pvkr" event={"ID":"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55","Type":"ContainerDied","Data":"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463"} Jul 09 07:10:59.002579 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.002573 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c645b97d7-7pvkr" event={"ID":"0ca8c21e-2e96-4dc0-a9ce-48b62f84be55","Type":"ContainerDied","Data":"4f34d2539efec2f830c3bdce20d151a45d8ca7604fdbae28a7202710ce616a42"} Jul 09 07:10:59.002694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.002602 2573 scope.go:117] "RemoveContainer" containerID="c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463" Jul 09 07:10:59.004691 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.004670 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" Jul 09 07:10:59.004794 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.004676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2" event={"ID":"30c0e91a-a16a-49bd-a8e3-40e7b3baf599","Type":"ContainerDied","Data":"1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5"} Jul 09 07:10:59.004794 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.004727 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d2bd26a514e76b52f27196c8925594f8a9f430555069411390e94bb2a5030e5" Jul 09 07:10:59.014327 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.014307 2573 scope.go:117] "RemoveContainer" containerID="c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463" Jul 09 07:10:59.014632 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:10:59.014613 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463\": container with ID starting with c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463 not found: ID does not exist" containerID="c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463" Jul 09 07:10:59.014722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.014640 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463"} err="failed to get container status \"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463\": rpc error: code = NotFound desc = could not find container \"c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463\": container with ID starting with c24991261ede03c8f37ea02062d5f0496bda921b999e0b216c45b5e807cfe463 not found: ID does not exist" Jul 09 07:10:59.035314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.035269 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:10:59.038541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.038513 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c645b97d7-7pvkr"] Jul 09 07:10:59.141582 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.141560 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:10:59.180848 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.180819 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kdcm\" (UniqueName: \"kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm\") pod \"ddb5e708-5449-4763-949a-d1da3d934776\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " Jul 09 07:10:59.181010 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.180869 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util\") pod \"ddb5e708-5449-4763-949a-d1da3d934776\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " Jul 09 07:10:59.181010 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.180966 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle\") pod \"ddb5e708-5449-4763-949a-d1da3d934776\" (UID: \"ddb5e708-5449-4763-949a-d1da3d934776\") " Jul 09 07:10:59.181370 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.181345 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle" (OuterVolumeSpecName: "bundle") pod "ddb5e708-5449-4763-949a-d1da3d934776" (UID: "ddb5e708-5449-4763-949a-d1da3d934776"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:59.182948 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.182920 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm" (OuterVolumeSpecName: "kube-api-access-2kdcm") pod "ddb5e708-5449-4763-949a-d1da3d934776" (UID: "ddb5e708-5449-4763-949a-d1da3d934776"). InnerVolumeSpecName "kube-api-access-2kdcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:10:59.186285 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.186263 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util" (OuterVolumeSpecName: "util") pod "ddb5e708-5449-4763-949a-d1da3d934776" (UID: "ddb5e708-5449-4763-949a-d1da3d934776"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:10:59.282413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.282297 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:59.282413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.282337 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kdcm\" (UniqueName: \"kubernetes.io/projected/ddb5e708-5449-4763-949a-d1da3d934776-kube-api-access-2kdcm\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:59.282413 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.282349 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddb5e708-5449-4763-949a-d1da3d934776-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:10:59.918168 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:10:59.918132 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" path="/var/lib/kubelet/pods/0ca8c21e-2e96-4dc0-a9ce-48b62f84be55/volumes" Jul 09 07:11:00.012249 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:00.012209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" event={"ID":"ddb5e708-5449-4763-949a-d1da3d934776","Type":"ContainerDied","Data":"a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10"} Jul 09 07:11:00.012632 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:00.012268 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a28dd003be2816d0e0031c899a8704abca7f98f7bd81a0c5cd3f710ae2b2df10" Jul 09 07:11:00.012632 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:00.012223 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45" Jul 09 07:11:05.598232 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:05.598196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:06.193737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:06.193705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:07.319553 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:07.319514 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:09.755666 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.755630 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4"] Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756018 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="util" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756034 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="util" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756047 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756055 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756067 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="extract" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756074 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="extract" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756086 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756095 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756104 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756111 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756118 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756126 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="pull" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756136 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="extract" Jul 09 07:11:09.756149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756143 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756157 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" containerName="console" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756165 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" containerName="console" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756175 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756183 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756194 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756201 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756216 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756226 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756238 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756246 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="util" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756255 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756263 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756342 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be316c7-df7b-478b-979e-6200b7ca5255" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756355 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c0e91a-a16a-49bd-a8e3-40e7b3baf599" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756365 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ca8c21e-2e96-4dc0-a9ce-48b62f84be55" containerName="console" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756374 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1853565c-d4f4-44cb-b698-5b0a4c1dd611" containerName="extract" Jul 09 07:11:09.756836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.756383 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb5e708-5449-4763-949a-d1da3d934776" containerName="extract" Jul 09 07:11:09.760637 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.760616 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:09.763662 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.763643 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xvvz7\"" Jul 09 07:11:09.787949 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.787916 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4"] Jul 09 07:11:09.855201 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.855166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcftq\" (UniqueName: \"kubernetes.io/projected/5675ae79-b653-4e49-80ce-c63881408de4-kube-api-access-rcftq\") pod \"authorino-operator-75cc8d5cb7-7mvq4\" (UID: \"5675ae79-b653-4e49-80ce-c63881408de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:09.955962 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.955925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcftq\" (UniqueName: \"kubernetes.io/projected/5675ae79-b653-4e49-80ce-c63881408de4-kube-api-access-rcftq\") pod \"authorino-operator-75cc8d5cb7-7mvq4\" (UID: \"5675ae79-b653-4e49-80ce-c63881408de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:09.987627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:09.987593 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcftq\" (UniqueName: \"kubernetes.io/projected/5675ae79-b653-4e49-80ce-c63881408de4-kube-api-access-rcftq\") pod \"authorino-operator-75cc8d5cb7-7mvq4\" (UID: \"5675ae79-b653-4e49-80ce-c63881408de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:10.070980 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:10.070893 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:10.225047 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:10.224841 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4"] Jul 09 07:11:10.227652 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:11:10.227625 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5675ae79_b653_4e49_80ce_c63881408de4.slice/crio-a31d8b86d6635d63feee241b15342aca9963cdf8b57fe91c417cda4a39dce95b WatchSource:0}: Error finding container a31d8b86d6635d63feee241b15342aca9963cdf8b57fe91c417cda4a39dce95b: Status 404 returned error can't find the container with id a31d8b86d6635d63feee241b15342aca9963cdf8b57fe91c417cda4a39dce95b Jul 09 07:11:10.902720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:10.902673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:11.063318 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:11.063288 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" event={"ID":"5675ae79-b653-4e49-80ce-c63881408de4","Type":"ContainerStarted","Data":"a31d8b86d6635d63feee241b15342aca9963cdf8b57fe91c417cda4a39dce95b"} Jul 09 07:11:13.285611 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.285576 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn"] Jul 09 07:11:13.287869 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.287848 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:13.293054 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.293023 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5xskm\"" Jul 09 07:11:13.293225 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.293208 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 07:11:13.309708 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.309673 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn"] Jul 09 07:11:13.385468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.384468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbbgh\" (UniqueName: \"kubernetes.io/projected/13b6ef49-736f-4d25-821a-2d547bbb4047-kube-api-access-mbbgh\") pod \"dns-operator-controller-manager-864dd78c4c-nz2hn\" (UID: \"13b6ef49-736f-4d25-821a-2d547bbb4047\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:13.485388 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.485353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbbgh\" (UniqueName: \"kubernetes.io/projected/13b6ef49-736f-4d25-821a-2d547bbb4047-kube-api-access-mbbgh\") pod \"dns-operator-controller-manager-864dd78c4c-nz2hn\" (UID: \"13b6ef49-736f-4d25-821a-2d547bbb4047\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:13.508716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.508680 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbbgh\" (UniqueName: \"kubernetes.io/projected/13b6ef49-736f-4d25-821a-2d547bbb4047-kube-api-access-mbbgh\") pod \"dns-operator-controller-manager-864dd78c4c-nz2hn\" (UID: \"13b6ef49-736f-4d25-821a-2d547bbb4047\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:13.597928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.597888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:13.668298 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.668260 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:13.741149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.741124 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn"] Jul 09 07:11:13.743024 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:11:13.743001 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b6ef49_736f_4d25_821a_2d547bbb4047.slice/crio-7b8c1d79c8be3125251100d451523d346a246efa44492acac0095ab4aa84c81c WatchSource:0}: Error finding container 7b8c1d79c8be3125251100d451523d346a246efa44492acac0095ab4aa84c81c: Status 404 returned error can't find the container with id 7b8c1d79c8be3125251100d451523d346a246efa44492acac0095ab4aa84c81c Jul 09 07:11:13.953109 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:13.953017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:14.081904 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:14.081864 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" event={"ID":"5675ae79-b653-4e49-80ce-c63881408de4","Type":"ContainerStarted","Data":"590fc87971d234401b4bc0a15dc1689c21cf89ad1d61fc45c65a87bfdbfb9f24"} Jul 09 07:11:14.082105 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:14.082020 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:14.083062 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:14.083039 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" event={"ID":"13b6ef49-736f-4d25-821a-2d547bbb4047","Type":"ContainerStarted","Data":"7b8c1d79c8be3125251100d451523d346a246efa44492acac0095ab4aa84c81c"} Jul 09 07:11:14.105158 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:14.105110 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" podStartSLOduration=2.207841026 podStartE2EDuration="5.105093413s" totalImagesPullingTime="2.897252387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:11:14.103328348 +0000 UTC m=+528.777185220" watchObservedRunningTime="2026-07-09 07:11:14.105093413 +0000 UTC m=+528.778950259" Jul 09 07:11:16.163118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:16.163080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:16.261372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:16.261324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:17.100200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:17.100114 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" event={"ID":"13b6ef49-736f-4d25-821a-2d547bbb4047","Type":"ContainerStarted","Data":"4f6c72c45931c36b650bc0c10bcb09ddf0454e2d01b9349b940de4d6471ac076"} Jul 09 07:11:17.100362 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:17.100216 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:17.126354 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:17.126300 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" podStartSLOduration=1.647245119 podStartE2EDuration="4.126283802s" totalImagesPullingTime="2.479038683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:11:17.12457336 +0000 UTC m=+531.798430228" watchObservedRunningTime="2026-07-09 07:11:17.126283802 +0000 UTC m=+531.800140649" Jul 09 07:11:18.771163 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:18.771130 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:20.080773 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.080740 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9"] Jul 09 07:11:20.083781 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.083765 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.093473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.093453 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gw8h4\"" Jul 09 07:11:20.109139 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.109102 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9"] Jul 09 07:11:20.151269 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.151239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq9kh\" (UniqueName: \"kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.151456 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.151361 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.252121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.252082 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq9kh\" (UniqueName: \"kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.252290 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.252187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.252535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.252518 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.263038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.263004 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq9kh\" (UniqueName: \"kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.393177 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.393147 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:20.585229 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:20.585207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9"] Jul 09 07:11:20.587822 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:11:20.587792 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2042ace_6a6e_49cc_ad3d_ac4526968dcd.slice/crio-e7f61c27a1eebb0a6377aff8bdc290d90930e43c122400b9b4a5f41f016b4819 WatchSource:0}: Error finding container e7f61c27a1eebb0a6377aff8bdc290d90930e43c122400b9b4a5f41f016b4819: Status 404 returned error can't find the container with id e7f61c27a1eebb0a6377aff8bdc290d90930e43c122400b9b4a5f41f016b4819 Jul 09 07:11:21.120204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:21.120171 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" event={"ID":"b2042ace-6a6e-49cc-ad3d-ac4526968dcd","Type":"ContainerStarted","Data":"e7f61c27a1eebb0a6377aff8bdc290d90930e43c122400b9b4a5f41f016b4819"} Jul 09 07:11:21.278598 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:21.278531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:21.395287 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:21.395244 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:21.537414 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:21.537347 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:21.692316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:21.692231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:25.046262 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:25.046232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:25.091128 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:25.091103 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7mvq4" Jul 09 07:11:25.147579 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:25.147540 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:28.106895 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:28.106859 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nz2hn" Jul 09 07:11:28.158951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:28.158901 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" event={"ID":"b2042ace-6a6e-49cc-ad3d-ac4526968dcd","Type":"ContainerStarted","Data":"c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506"} Jul 09 07:11:28.159147 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:28.159127 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:28.178466 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:28.178385 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" podStartSLOduration=1.385629288 podStartE2EDuration="8.178370505s" totalImagesPullingTime="6.792741217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:11:28.177135921 +0000 UTC m=+542.850992846" watchObservedRunningTime="2026-07-09 07:11:28.178370505 +0000 UTC m=+542.852227351" Jul 09 07:11:29.105624 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:29.105588 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:29.314173 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:29.314142 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:30.171127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:30.171097 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:30.593583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:30.593498 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:32.654690 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:32.654659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:32.757464 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:32.757432 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:38.023945 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:38.023911 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:38.117283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:38.117247 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:39.167862 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:39.167831 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:40.734610 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:40.734550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:40.929604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:40.929569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:41.165493 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.165453 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:41.272852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.272816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:41.275458 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.275419 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9"] Jul 09 07:11:41.275702 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.275679 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" containerName="manager" containerID="cri-o://c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506" gracePeriod=2 Jul 09 07:11:41.284781 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.284758 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9"] Jul 09 07:11:41.339739 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.339708 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:11:41.340079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.340064 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" containerName="manager" Jul 09 07:11:41.340165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.340083 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" containerName="manager" Jul 09 07:11:41.340165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.340161 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" containerName="manager" Jul 09 07:11:41.343440 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.343419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.345617 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:11:41.345591 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:41.380312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.380286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:11:41.437895 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.437803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.437895 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.437851 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.527616 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.527590 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:41.530056 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:11:41.530032 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9pl9\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:41.538450 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.538423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.538527 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.538465 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.538860 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.538843 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.552139 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.552112 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mvd74\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.639287 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.639250 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq9kh\" (UniqueName: \"kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh\") pod \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " Jul 09 07:11:41.639509 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.639330 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume\") pod \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\" (UID: \"b2042ace-6a6e-49cc-ad3d-ac4526968dcd\") " Jul 09 07:11:41.639823 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.639796 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2042ace-6a6e-49cc-ad3d-ac4526968dcd" (UID: "b2042ace-6a6e-49cc-ad3d-ac4526968dcd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:11:41.641416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.641381 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh" (OuterVolumeSpecName: "kube-api-access-gq9kh") pod "b2042ace-6a6e-49cc-ad3d-ac4526968dcd" (UID: "b2042ace-6a6e-49cc-ad3d-ac4526968dcd"). InnerVolumeSpecName "kube-api-access-gq9kh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:11:41.662710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.662676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:41.740565 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.740534 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-extensions-socket-volume\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:11:41.740565 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.740563 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq9kh\" (UniqueName: \"kubernetes.io/projected/b2042ace-6a6e-49cc-ad3d-ac4526968dcd-kube-api-access-gq9kh\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:11:41.810349 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.810324 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:11:41.812845 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:11:41.812823 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9e92a7_712f_4ca0_b041_fac9954792ae.slice/crio-3486f72a27db469b2d5efb715aefd335648aba38540db96bf0275af4283d157a WatchSource:0}: Error finding container 3486f72a27db469b2d5efb715aefd335648aba38540db96bf0275af4283d157a: Status 404 returned error can't find the container with id 3486f72a27db469b2d5efb715aefd335648aba38540db96bf0275af4283d157a Jul 09 07:11:41.917940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:41.917904 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" path="/var/lib/kubelet/pods/b2042ace-6a6e-49cc-ad3d-ac4526968dcd/volumes" Jul 09 07:11:42.229639 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.229599 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" event={"ID":"ab9e92a7-712f-4ca0-b041-fac9954792ae","Type":"ContainerStarted","Data":"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804"} Jul 09 07:11:42.229928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.229905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" event={"ID":"ab9e92a7-712f-4ca0-b041-fac9954792ae","Type":"ContainerStarted","Data":"3486f72a27db469b2d5efb715aefd335648aba38540db96bf0275af4283d157a"} Jul 09 07:11:42.230083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.230062 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:11:42.232494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.232017 2573 generic.go:356] "Generic (PLEG): container finished" podID="b2042ace-6a6e-49cc-ad3d-ac4526968dcd" containerID="c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506" exitCode=0 Jul 09 07:11:42.232494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.232088 2573 scope.go:117] "RemoveContainer" containerID="c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506" Jul 09 07:11:42.232494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.232230 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9pl9" Jul 09 07:11:42.242018 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.242001 2573 scope.go:117] "RemoveContainer" containerID="c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506" Jul 09 07:11:42.242259 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:11:42.242239 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506\": container with ID starting with c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506 not found: ID does not exist" containerID="c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506" Jul 09 07:11:42.242320 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.242268 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506"} err="failed to get container status \"c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506\": rpc error: code = NotFound desc = could not find container \"c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506\": container with ID starting with c817cda21930909416fa49067cb8c69b883ca81c82112841d6ab4a2a2c4fc506 not found: ID does not exist" Jul 09 07:11:42.258440 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:42.258380 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" podStartSLOduration=1.258366205 podStartE2EDuration="1.258366205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:11:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:11:42.257256134 +0000 UTC m=+556.931113004" watchObservedRunningTime="2026-07-09 07:11:42.258366205 +0000 UTC m=+556.932223052" Jul 09 07:11:44.885359 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:44.885330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:45.000445 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:45.000383 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:47.908509 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:47.908471 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:48.018840 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:48.018805 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:51.549945 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:51.549908 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:51.653055 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:51.653019 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:11:53.241297 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:11:53.241269 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:12:00.696150 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:00.696111 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:12:00.696635 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:00.696334 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" podUID="ab9e92a7-712f-4ca0-b041-fac9954792ae" containerName="manager" containerID="cri-o://9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804" gracePeriod=10 Jul 09 07:12:00.978421 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:00.978340 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:12:01.005416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.005358 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq\") pod \"ab9e92a7-712f-4ca0-b041-fac9954792ae\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " Jul 09 07:12:01.005589 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.005428 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume\") pod \"ab9e92a7-712f-4ca0-b041-fac9954792ae\" (UID: \"ab9e92a7-712f-4ca0-b041-fac9954792ae\") " Jul 09 07:12:01.005810 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.005787 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ab9e92a7-712f-4ca0-b041-fac9954792ae" (UID: "ab9e92a7-712f-4ca0-b041-fac9954792ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 07:12:01.007457 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.007436 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq" (OuterVolumeSpecName: "kube-api-access-2p2jq") pod "ab9e92a7-712f-4ca0-b041-fac9954792ae" (UID: "ab9e92a7-712f-4ca0-b041-fac9954792ae"). InnerVolumeSpecName "kube-api-access-2p2jq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:01.106494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.106457 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p2jq\" (UniqueName: \"kubernetes.io/projected/ab9e92a7-712f-4ca0-b041-fac9954792ae-kube-api-access-2p2jq\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:01.106494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.106490 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab9e92a7-712f-4ca0-b041-fac9954792ae-extensions-socket-volume\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:01.332717 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.332631 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab9e92a7-712f-4ca0-b041-fac9954792ae" containerID="9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804" exitCode=0 Jul 09 07:12:01.332717 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.332682 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" Jul 09 07:12:01.332905 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.332722 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" event={"ID":"ab9e92a7-712f-4ca0-b041-fac9954792ae","Type":"ContainerDied","Data":"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804"} Jul 09 07:12:01.332905 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.332781 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74" event={"ID":"ab9e92a7-712f-4ca0-b041-fac9954792ae","Type":"ContainerDied","Data":"3486f72a27db469b2d5efb715aefd335648aba38540db96bf0275af4283d157a"} Jul 09 07:12:01.332905 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.332808 2573 scope.go:117] "RemoveContainer" containerID="9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804" Jul 09 07:12:01.341634 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.341616 2573 scope.go:117] "RemoveContainer" containerID="9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804" Jul 09 07:12:01.341888 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:01.341868 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804\": container with ID starting with 9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804 not found: ID does not exist" containerID="9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804" Jul 09 07:12:01.341955 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.341902 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804"} err="failed to get container status \"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804\": rpc error: code = NotFound desc = could not find container \"9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804\": container with ID starting with 9bddd7cec96d0fb2f4a76a4161543a1e1fd4f8516097f5960e9bdf62e24ac804 not found: ID does not exist" Jul 09 07:12:01.365269 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.365238 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:12:01.377588 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.377557 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mvd74"] Jul 09 07:12:01.916796 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:01.916761 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9e92a7-712f-4ca0-b041-fac9954792ae" path="/var/lib/kubelet/pods/ab9e92a7-712f-4ca0-b041-fac9954792ae/volumes" Jul 09 07:12:07.704652 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:07.704615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:15.332422 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:15.332367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:17.185919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.185883 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr"] Jul 09 07:12:17.186375 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.186342 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9e92a7-712f-4ca0-b041-fac9954792ae" containerName="manager" Jul 09 07:12:17.186375 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.186360 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9e92a7-712f-4ca0-b041-fac9954792ae" containerName="manager" Jul 09 07:12:17.186538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.186454 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9e92a7-712f-4ca0-b041-fac9954792ae" containerName="manager" Jul 09 07:12:17.189115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.189093 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.193798 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.193781 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lg5cw\"" Jul 09 07:12:17.211209 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.211184 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr"] Jul 09 07:12:17.220496 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220504 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvx9m\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-kube-api-access-rvx9m\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220782 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220722 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220782 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.220851 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.220789 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.223114 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.223090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:17.322043 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322061 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvx9m\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-kube-api-access-rvx9m\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322138 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322173 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322206 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322695 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322695 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322592 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.322836 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.323004 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.322981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.324485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.324466 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.324754 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.324735 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.330968 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.330943 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.331238 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.331212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvx9m\" (UniqueName: \"kubernetes.io/projected/6d1f9a59-576b-40f6-bd3a-98fcf6249a1a-kube-api-access-rvx9m\") pod \"maas-default-gateway-openshift-default-56654dcf7-mfbgr\" (UID: \"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.418715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.418685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:17.499554 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.499460 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:17.632081 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.632050 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr"] Jul 09 07:12:17.633788 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:17.633762 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1f9a59_576b_40f6_bd3a_98fcf6249a1a.slice/crio-9204a7ca0509796949af83dc5b8c622ef6f883df9018b2d513f1751876eae8c7 WatchSource:0}: Error finding container 9204a7ca0509796949af83dc5b8c622ef6f883df9018b2d513f1751876eae8c7: Status 404 returned error can't find the container with id 9204a7ca0509796949af83dc5b8c622ef6f883df9018b2d513f1751876eae8c7 Jul 09 07:12:17.636326 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.636289 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:12:17.636448 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.636359 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:12:17.636448 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:17.636437 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:12:18.416444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:18.416402 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" event={"ID":"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a","Type":"ContainerStarted","Data":"410cecd64e8b03b366d5cf785c66481c54d5fd50c52216b9fed7e241b8c2ba03"} Jul 09 07:12:18.416801 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:18.416466 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" event={"ID":"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a","Type":"ContainerStarted","Data":"9204a7ca0509796949af83dc5b8c622ef6f883df9018b2d513f1751876eae8c7"} Jul 09 07:12:18.436756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:18.436668 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" podStartSLOduration=1.436649998 podStartE2EDuration="1.436649998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:18.435148752 +0000 UTC m=+593.109005623" watchObservedRunningTime="2026-07-09 07:12:18.436649998 +0000 UTC m=+593.110506846" Jul 09 07:12:18.500635 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:18.500598 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:18.505370 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:18.505347 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:19.421234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.421201 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:19.422439 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.422417 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:12:19.759957 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.759875 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:19.762534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.762517 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.764594 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.764569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:19.765725 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.765708 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 07:12:19.765886 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.765867 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9hnnf\"" Jul 09 07:12:19.773775 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.773753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:19.839828 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.839790 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxx4\" (UniqueName: \"kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.840010 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.839906 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.852780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.852751 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:19.870442 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.870412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:19.940806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.940778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxx4\" (UniqueName: \"kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.940974 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.940847 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.941449 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.941430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.953162 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.953137 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxx4\" (UniqueName: \"kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4\") pod \"limitador-limitador-64c94fd64f-g8tj7\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:19.971500 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:19.971471 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:20.073465 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.073354 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:20.215856 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.215824 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:20.217189 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:20.217165 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87aa60ce_83ef_45d3_87f9_eac8ce071860.slice/crio-8da42d4f5c9ff1d282e77de49ab5cdce8defa20dca425cef43d2d5348b8965f2 WatchSource:0}: Error finding container 8da42d4f5c9ff1d282e77de49ab5cdce8defa20dca425cef43d2d5348b8965f2: Status 404 returned error can't find the container with id 8da42d4f5c9ff1d282e77de49ab5cdce8defa20dca425cef43d2d5348b8965f2 Jul 09 07:12:20.426705 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.426669 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" event={"ID":"87aa60ce-83ef-45d3-87f9-eac8ce071860","Type":"ContainerStarted","Data":"8da42d4f5c9ff1d282e77de49ab5cdce8defa20dca425cef43d2d5348b8965f2"} Jul 09 07:12:20.633316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.633277 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:20.636129 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.636107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:20.638962 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.638946 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-k4czs\"" Jul 09 07:12:20.644897 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.644871 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:20.646066 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.646044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcgv\" (UniqueName: \"kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv\") pod \"authorino-58cd955675-9bvtw\" (UID: \"1cfa4d59-8e5e-483e-b56c-20ca261dfafb\") " pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:20.674917 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.674895 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:20.746669 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.746590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcgv\" (UniqueName: \"kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv\") pod \"authorino-58cd955675-9bvtw\" (UID: \"1cfa4d59-8e5e-483e-b56c-20ca261dfafb\") " pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:20.762101 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.762073 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcgv\" (UniqueName: \"kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv\") pod \"authorino-58cd955675-9bvtw\" (UID: \"1cfa4d59-8e5e-483e-b56c-20ca261dfafb\") " pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:20.795375 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.795345 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:20.833451 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.833420 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:20.836321 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.836305 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:20.847212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.847191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlnc\" (UniqueName: \"kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc\") pod \"authorino-75d556bd77-p9tfh\" (UID: \"7e36e9b7-6f15-4de8-9317-5f110de5838f\") " pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:20.856535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.856513 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:20.946463 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.946432 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:20.947745 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.947713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlnc\" (UniqueName: \"kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc\") pod \"authorino-75d556bd77-p9tfh\" (UID: \"7e36e9b7-6f15-4de8-9317-5f110de5838f\") " pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:20.957779 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:20.957748 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlnc\" (UniqueName: \"kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc\") pod \"authorino-75d556bd77-p9tfh\" (UID: \"7e36e9b7-6f15-4de8-9317-5f110de5838f\") " pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:21.116834 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:21.116809 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:21.119904 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:21.119873 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cfa4d59_8e5e_483e_b56c_20ca261dfafb.slice/crio-40696b47b6bc8bccd2a9e671fd37558dc062dec65f3d550829178b3ce62f534e WatchSource:0}: Error finding container 40696b47b6bc8bccd2a9e671fd37558dc062dec65f3d550829178b3ce62f534e: Status 404 returned error can't find the container with id 40696b47b6bc8bccd2a9e671fd37558dc062dec65f3d550829178b3ce62f534e Jul 09 07:12:21.145889 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:21.145858 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:21.321019 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:21.320949 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:21.322753 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:21.322726 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e36e9b7_6f15_4de8_9317_5f110de5838f.slice/crio-cf16650112e2b72a6d8b674c1d7c4be56ab0014f36bd1d70e69c5a9e94f263e9 WatchSource:0}: Error finding container cf16650112e2b72a6d8b674c1d7c4be56ab0014f36bd1d70e69c5a9e94f263e9: Status 404 returned error can't find the container with id cf16650112e2b72a6d8b674c1d7c4be56ab0014f36bd1d70e69c5a9e94f263e9 Jul 09 07:12:21.434096 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:21.434055 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9bvtw" event={"ID":"1cfa4d59-8e5e-483e-b56c-20ca261dfafb","Type":"ContainerStarted","Data":"40696b47b6bc8bccd2a9e671fd37558dc062dec65f3d550829178b3ce62f534e"} Jul 09 07:12:21.435469 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:21.435423 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p9tfh" event={"ID":"7e36e9b7-6f15-4de8-9317-5f110de5838f","Type":"ContainerStarted","Data":"cf16650112e2b72a6d8b674c1d7c4be56ab0014f36bd1d70e69c5a9e94f263e9"} Jul 09 07:12:25.475332 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.475250 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" event={"ID":"87aa60ce-83ef-45d3-87f9-eac8ce071860","Type":"ContainerStarted","Data":"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4"} Jul 09 07:12:25.475757 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.475335 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:25.477132 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.477107 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p9tfh" event={"ID":"7e36e9b7-6f15-4de8-9317-5f110de5838f","Type":"ContainerStarted","Data":"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8"} Jul 09 07:12:25.478592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.478566 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9bvtw" event={"ID":"1cfa4d59-8e5e-483e-b56c-20ca261dfafb","Type":"ContainerStarted","Data":"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b"} Jul 09 07:12:25.493035 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.492994 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" podStartSLOduration=1.521579858 podStartE2EDuration="6.492979452s" totalImagesPullingTime="4.971399594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:25.491952559 +0000 UTC m=+600.165809419" watchObservedRunningTime="2026-07-09 07:12:25.492979452 +0000 UTC m=+600.166836299" Jul 09 07:12:25.506925 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.506885 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-9bvtw" podStartSLOduration=1.374913383 podStartE2EDuration="5.506869001s" totalImagesPullingTime="4.131955618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:25.505235945 +0000 UTC m=+600.179092795" watchObservedRunningTime="2026-07-09 07:12:25.506869001 +0000 UTC m=+600.180725847" Jul 09 07:12:25.524950 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.524904 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-p9tfh" podStartSLOduration=1.582515251 podStartE2EDuration="5.524886317s" totalImagesPullingTime="3.942371066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:25.522492433 +0000 UTC m=+600.196349283" watchObservedRunningTime="2026-07-09 07:12:25.524886317 +0000 UTC m=+600.198743164" Jul 09 07:12:25.557247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.557216 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:25.755622 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.755544 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:12:25.755787 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:25.755728 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:12:27.488350 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:27.488311 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-9bvtw" podUID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" containerName="authorino" containerID="cri-o://eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b" gracePeriod=30 Jul 09 07:12:27.734262 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:27.734238 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:27.809733 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:27.809650 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtcgv\" (UniqueName: \"kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv\") pod \"1cfa4d59-8e5e-483e-b56c-20ca261dfafb\" (UID: \"1cfa4d59-8e5e-483e-b56c-20ca261dfafb\") " Jul 09 07:12:27.811765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:27.811731 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv" (OuterVolumeSpecName: "kube-api-access-gtcgv") pod "1cfa4d59-8e5e-483e-b56c-20ca261dfafb" (UID: "1cfa4d59-8e5e-483e-b56c-20ca261dfafb"). InnerVolumeSpecName "kube-api-access-gtcgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:27.910810 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:27.910774 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtcgv\" (UniqueName: \"kubernetes.io/projected/1cfa4d59-8e5e-483e-b56c-20ca261dfafb-kube-api-access-gtcgv\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:28.494858 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.494824 2573 generic.go:356] "Generic (PLEG): container finished" podID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" containerID="eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b" exitCode=0 Jul 09 07:12:28.494858 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.494862 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9bvtw" event={"ID":"1cfa4d59-8e5e-483e-b56c-20ca261dfafb","Type":"ContainerDied","Data":"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b"} Jul 09 07:12:28.495338 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.494875 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9bvtw" Jul 09 07:12:28.495338 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.494890 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9bvtw" event={"ID":"1cfa4d59-8e5e-483e-b56c-20ca261dfafb","Type":"ContainerDied","Data":"40696b47b6bc8bccd2a9e671fd37558dc062dec65f3d550829178b3ce62f534e"} Jul 09 07:12:28.495338 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.494906 2573 scope.go:117] "RemoveContainer" containerID="eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b" Jul 09 07:12:28.503358 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.503341 2573 scope.go:117] "RemoveContainer" containerID="eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b" Jul 09 07:12:28.503631 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:28.503607 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b\": container with ID starting with eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b not found: ID does not exist" containerID="eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b" Jul 09 07:12:28.503696 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.503640 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b"} err="failed to get container status \"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b\": rpc error: code = NotFound desc = could not find container \"eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b\": container with ID starting with eaec9f8148c7c7f1030be8ffc0138054679049f39211861ae8bd7fd5ba8fd71b not found: ID does not exist" Jul 09 07:12:28.522453 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.522420 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:28.525906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:28.525882 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-9bvtw"] Jul 09 07:12:29.919042 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:29.918996 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" path="/var/lib/kubelet/pods/1cfa4d59-8e5e-483e-b56c-20ca261dfafb/volumes" Jul 09 07:12:35.422797 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:35.422767 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:35.423165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:35.422982 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" podUID="87aa60ce-83ef-45d3-87f9-eac8ce071860" containerName="limitador" containerID="cri-o://ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4" gracePeriod=30 Jul 09 07:12:35.423674 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:35.423606 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:35.976604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:35.976574 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:36.079234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.079145 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsxx4\" (UniqueName: \"kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4\") pod \"87aa60ce-83ef-45d3-87f9-eac8ce071860\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " Jul 09 07:12:36.079234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.079195 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file\") pod \"87aa60ce-83ef-45d3-87f9-eac8ce071860\" (UID: \"87aa60ce-83ef-45d3-87f9-eac8ce071860\") " Jul 09 07:12:36.079590 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.079556 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file" (OuterVolumeSpecName: "config-file") pod "87aa60ce-83ef-45d3-87f9-eac8ce071860" (UID: "87aa60ce-83ef-45d3-87f9-eac8ce071860"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:12:36.081243 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.081222 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4" (OuterVolumeSpecName: "kube-api-access-zsxx4") pod "87aa60ce-83ef-45d3-87f9-eac8ce071860" (UID: "87aa60ce-83ef-45d3-87f9-eac8ce071860"). InnerVolumeSpecName "kube-api-access-zsxx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:36.180314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.180279 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsxx4\" (UniqueName: \"kubernetes.io/projected/87aa60ce-83ef-45d3-87f9-eac8ce071860-kube-api-access-zsxx4\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:36.180314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.180309 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/87aa60ce-83ef-45d3-87f9-eac8ce071860-config-file\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:36.535148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.535107 2573 generic.go:356] "Generic (PLEG): container finished" podID="87aa60ce-83ef-45d3-87f9-eac8ce071860" containerID="ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4" exitCode=0 Jul 09 07:12:36.535608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.535156 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" Jul 09 07:12:36.535608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.535197 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" event={"ID":"87aa60ce-83ef-45d3-87f9-eac8ce071860","Type":"ContainerDied","Data":"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4"} Jul 09 07:12:36.535608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.535297 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-g8tj7" event={"ID":"87aa60ce-83ef-45d3-87f9-eac8ce071860","Type":"ContainerDied","Data":"8da42d4f5c9ff1d282e77de49ab5cdce8defa20dca425cef43d2d5348b8965f2"} Jul 09 07:12:36.535608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.535330 2573 scope.go:117] "RemoveContainer" containerID="ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4" Jul 09 07:12:36.544094 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.544078 2573 scope.go:117] "RemoveContainer" containerID="ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4" Jul 09 07:12:36.544337 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:36.544316 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4\": container with ID starting with ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4 not found: ID does not exist" containerID="ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4" Jul 09 07:12:36.544418 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.544344 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4"} err="failed to get container status \"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4\": rpc error: code = NotFound desc = could not find container \"ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4\": container with ID starting with ca7b30cb9f545cad97e0fd33182e206821f9361849d08bfbea71d7ad6e20c9b4 not found: ID does not exist" Jul 09 07:12:36.555555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.555533 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:36.563007 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:36.562984 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-g8tj7"] Jul 09 07:12:37.896785 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:37.896753 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:37.916809 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:37.916780 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87aa60ce-83ef-45d3-87f9-eac8ce071860" path="/var/lib/kubelet/pods/87aa60ce-83ef-45d3-87f9-eac8ce071860/volumes" Jul 09 07:12:40.843734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:40.843705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:40.941062 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:40.941033 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:41.336704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.336670 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p97r5"] Jul 09 07:12:41.336991 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.336979 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87aa60ce-83ef-45d3-87f9-eac8ce071860" containerName="limitador" Jul 09 07:12:41.337034 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.336992 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="87aa60ce-83ef-45d3-87f9-eac8ce071860" containerName="limitador" Jul 09 07:12:41.337034 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.337006 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" containerName="authorino" Jul 09 07:12:41.337034 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.337011 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" containerName="authorino" Jul 09 07:12:41.337125 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.337061 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="87aa60ce-83ef-45d3-87f9-eac8ce071860" containerName="limitador" Jul 09 07:12:41.337125 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.337068 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cfa4d59-8e5e-483e-b56c-20ca261dfafb" containerName="authorino" Jul 09 07:12:41.341159 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.341141 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.343858 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.343832 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 07:12:41.343858 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.343856 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xj888\"" Jul 09 07:12:41.344061 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.343832 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 07:12:41.344656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.344626 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 07:12:41.351055 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.351028 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p97r5"] Jul 09 07:12:41.370094 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.370065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:41.427405 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.427363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-data\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.427592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.427445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rgjf\" (UniqueName: \"kubernetes.io/projected/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-kube-api-access-4rgjf\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.527870 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.527823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-data\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.528021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.527902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rgjf\" (UniqueName: \"kubernetes.io/projected/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-kube-api-access-4rgjf\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.528313 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.528293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-data\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.536051 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.536031 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rgjf\" (UniqueName: \"kubernetes.io/projected/daefd39d-6aad-4a1c-87dd-43ac72f81aa4-kube-api-access-4rgjf\") pod \"postgres-69fb4594cf-p97r5\" (UID: \"daefd39d-6aad-4a1c-87dd-43ac72f81aa4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.652716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.652688 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:41.789963 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:41.789931 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p97r5"] Jul 09 07:12:41.791708 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:41.791686 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaefd39d_6aad_4a1c_87dd_43ac72f81aa4.slice/crio-498f742595084072b4fbad480935ecdab0ac7bbc1efb63eda9b571d5edb1fd08 WatchSource:0}: Error finding container 498f742595084072b4fbad480935ecdab0ac7bbc1efb63eda9b571d5edb1fd08: Status 404 returned error can't find the container with id 498f742595084072b4fbad480935ecdab0ac7bbc1efb63eda9b571d5edb1fd08 Jul 09 07:12:42.566818 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:42.566783 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" event={"ID":"daefd39d-6aad-4a1c-87dd-43ac72f81aa4","Type":"ContainerStarted","Data":"498f742595084072b4fbad480935ecdab0ac7bbc1efb63eda9b571d5edb1fd08"} Jul 09 07:12:47.597424 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:47.597371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" event={"ID":"daefd39d-6aad-4a1c-87dd-43ac72f81aa4","Type":"ContainerStarted","Data":"4177b49055aed95850906b967bc3eced69104abb4d1d6c34947d46338bd2d460"} Jul 09 07:12:47.599345 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:47.597481 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:47.614808 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:47.614761 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" podStartSLOduration=1.575481006 podStartE2EDuration="6.614746397s" totalImagesPullingTime="5.039265391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:47.61328219 +0000 UTC m=+622.287139037" watchObservedRunningTime="2026-07-09 07:12:47.614746397 +0000 UTC m=+622.288603243" Jul 09 07:12:53.632796 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:53.632766 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p97r5" Jul 09 07:12:53.954372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:53.954317 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:54.192661 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.192628 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8qppz"] Jul 09 07:12:54.198273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.198256 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.203091 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.203064 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8qppz"] Jul 09 07:12:54.342724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.342684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgsg\" (UniqueName: \"kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg\") pod \"authorino-5d4b67c446-8qppz\" (UID: \"8749b78c-d96d-4054-a50a-42bb205917e8\") " pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.440408 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.440361 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8qppz"] Jul 09 07:12:54.440680 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:54.440657 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6qgsg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-8qppz" podUID="8749b78c-d96d-4054-a50a-42bb205917e8" Jul 09 07:12:54.443334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.443309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgsg\" (UniqueName: \"kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg\") pod \"authorino-5d4b67c446-8qppz\" (UID: \"8749b78c-d96d-4054-a50a-42bb205917e8\") " pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.451862 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.451840 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgsg\" (UniqueName: \"kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg\") pod \"authorino-5d4b67c446-8qppz\" (UID: \"8749b78c-d96d-4054-a50a-42bb205917e8\") " pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.472549 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.472478 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c45fd68dd-gr89l"] Jul 09 07:12:54.474861 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.474841 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.482348 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.482325 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c45fd68dd-gr89l"] Jul 09 07:12:54.505686 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.505654 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c45fd68dd-gr89l"] Jul 09 07:12:54.505930 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:54.505907 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wkvxs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-c45fd68dd-gr89l" podUID="76cce730-4db0-4079-90ad-a2ddec2b4c97" Jul 09 07:12:54.537323 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.537295 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:12:54.539672 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.539658 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.542011 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.541992 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 07:12:54.543751 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.543725 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkvxs\" (UniqueName: \"kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs\") pod \"authorino-c45fd68dd-gr89l\" (UID: \"76cce730-4db0-4079-90ad-a2ddec2b4c97\") " pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.547956 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.547937 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:12:54.632243 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.632209 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.632412 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.632213 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.640936 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.640917 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.641915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.641899 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:54.644278 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.644262 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkvxs\" (UniqueName: \"kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs\") pod \"authorino-c45fd68dd-gr89l\" (UID: \"76cce730-4db0-4079-90ad-a2ddec2b4c97\") " pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.644335 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.644323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.644383 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.644344 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjtcv\" (UniqueName: \"kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.655149 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.655127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkvxs\" (UniqueName: \"kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs\") pod \"authorino-c45fd68dd-gr89l\" (UID: \"76cce730-4db0-4079-90ad-a2ddec2b4c97\") " pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:54.745640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.745563 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qgsg\" (UniqueName: \"kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg\") pod \"8749b78c-d96d-4054-a50a-42bb205917e8\" (UID: \"8749b78c-d96d-4054-a50a-42bb205917e8\") " Jul 09 07:12:54.745640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.745618 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkvxs\" (UniqueName: \"kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs\") pod \"76cce730-4db0-4079-90ad-a2ddec2b4c97\" (UID: \"76cce730-4db0-4079-90ad-a2ddec2b4c97\") " Jul 09 07:12:54.745852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.745829 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.745904 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.745872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjtcv\" (UniqueName: \"kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.747784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.747753 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs" (OuterVolumeSpecName: "kube-api-access-wkvxs") pod "76cce730-4db0-4079-90ad-a2ddec2b4c97" (UID: "76cce730-4db0-4079-90ad-a2ddec2b4c97"). InnerVolumeSpecName "kube-api-access-wkvxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:54.747930 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.747821 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg" (OuterVolumeSpecName: "kube-api-access-6qgsg") pod "8749b78c-d96d-4054-a50a-42bb205917e8" (UID: "8749b78c-d96d-4054-a50a-42bb205917e8"). InnerVolumeSpecName "kube-api-access-6qgsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:54.748210 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.748189 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.753530 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.753462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjtcv\" (UniqueName: \"kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv\") pod \"authorino-5c979fd5f6-vpfph\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.847192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.847160 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qgsg\" (UniqueName: \"kubernetes.io/projected/8749b78c-d96d-4054-a50a-42bb205917e8-kube-api-access-6qgsg\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:54.847192 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.847184 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkvxs\" (UniqueName: \"kubernetes.io/projected/76cce730-4db0-4079-90ad-a2ddec2b4c97-kube-api-access-wkvxs\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:54.849055 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.849034 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:12:54.979179 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.979152 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:12:54.980172 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:12:54.980145 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a25647f_d893_4f9d_bb21_a5612e7da60c.slice/crio-b632ad06b16431ab93b3c6956e07a662dfbfcf0b37c0f9bcc468106888a93101 WatchSource:0}: Error finding container b632ad06b16431ab93b3c6956e07a662dfbfcf0b37c0f9bcc468106888a93101: Status 404 returned error can't find the container with id b632ad06b16431ab93b3c6956e07a662dfbfcf0b37c0f9bcc468106888a93101 Jul 09 07:12:54.981439 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:54.981425 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:12:55.242083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.242049 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:12:55.641670 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.641637 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" event={"ID":"8a25647f-d893-4f9d-bb21-a5612e7da60c","Type":"ContainerStarted","Data":"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7"} Jul 09 07:12:55.642102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.641683 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" event={"ID":"8a25647f-d893-4f9d-bb21-a5612e7da60c","Type":"ContainerStarted","Data":"b632ad06b16431ab93b3c6956e07a662dfbfcf0b37c0f9bcc468106888a93101"} Jul 09 07:12:55.642102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.641684 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8qppz" Jul 09 07:12:55.642102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.641700 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c45fd68dd-gr89l" Jul 09 07:12:55.657646 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.657603 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" podStartSLOduration=1.27514966 podStartE2EDuration="1.657587583s" totalImagesPullingTime="382.437923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:12:55.65708252 +0000 UTC m=+630.330939368" watchObservedRunningTime="2026-07-09 07:12:55.657587583 +0000 UTC m=+630.331444431" Jul 09 07:12:55.686612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.686585 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:55.686793 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.686767 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-p9tfh" podUID="7e36e9b7-6f15-4de8-9317-5f110de5838f" containerName="authorino" containerID="cri-o://6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8" gracePeriod=30 Jul 09 07:12:55.690292 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.690266 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8qppz"] Jul 09 07:12:55.701542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.701515 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8qppz"] Jul 09 07:12:55.729361 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.729333 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c45fd68dd-gr89l"] Jul 09 07:12:55.733334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.733287 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c45fd68dd-gr89l"] Jul 09 07:12:55.919352 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.919325 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76cce730-4db0-4079-90ad-a2ddec2b4c97" path="/var/lib/kubelet/pods/76cce730-4db0-4079-90ad-a2ddec2b4c97/volumes" Jul 09 07:12:55.919757 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.919738 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8749b78c-d96d-4054-a50a-42bb205917e8" path="/var/lib/kubelet/pods/8749b78c-d96d-4054-a50a-42bb205917e8/volumes" Jul 09 07:12:55.932066 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:55.932045 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:56.059383 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.059336 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwlnc\" (UniqueName: \"kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc\") pod \"7e36e9b7-6f15-4de8-9317-5f110de5838f\" (UID: \"7e36e9b7-6f15-4de8-9317-5f110de5838f\") " Jul 09 07:12:56.061380 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.061351 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc" (OuterVolumeSpecName: "kube-api-access-rwlnc") pod "7e36e9b7-6f15-4de8-9317-5f110de5838f" (UID: "7e36e9b7-6f15-4de8-9317-5f110de5838f"). InnerVolumeSpecName "kube-api-access-rwlnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:12:56.160366 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.160331 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwlnc\" (UniqueName: \"kubernetes.io/projected/7e36e9b7-6f15-4de8-9317-5f110de5838f-kube-api-access-rwlnc\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:12:56.647544 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.647505 2573 generic.go:356] "Generic (PLEG): container finished" podID="7e36e9b7-6f15-4de8-9317-5f110de5838f" containerID="6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8" exitCode=0 Jul 09 07:12:56.647989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.647558 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p9tfh" Jul 09 07:12:56.647989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.647587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p9tfh" event={"ID":"7e36e9b7-6f15-4de8-9317-5f110de5838f","Type":"ContainerDied","Data":"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8"} Jul 09 07:12:56.647989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.647639 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p9tfh" event={"ID":"7e36e9b7-6f15-4de8-9317-5f110de5838f","Type":"ContainerDied","Data":"cf16650112e2b72a6d8b674c1d7c4be56ab0014f36bd1d70e69c5a9e94f263e9"} Jul 09 07:12:56.647989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.647663 2573 scope.go:117] "RemoveContainer" containerID="6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8" Jul 09 07:12:56.657204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.657193 2573 scope.go:117] "RemoveContainer" containerID="6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8" Jul 09 07:12:56.657465 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:12:56.657438 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8\": container with ID starting with 6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8 not found: ID does not exist" containerID="6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8" Jul 09 07:12:56.657558 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.657481 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8"} err="failed to get container status \"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8\": rpc error: code = NotFound desc = could not find container \"6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8\": container with ID starting with 6d7517374af61c48fcde2dfbe2fdfb675a7e1bad7c570bfd25645f70dcf658e8 not found: ID does not exist" Jul 09 07:12:56.669137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.669111 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:56.673387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:56.673368 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p9tfh"] Jul 09 07:12:57.917786 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:12:57.917754 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e36e9b7-6f15-4de8-9317-5f110de5838f" path="/var/lib/kubelet/pods/7e36e9b7-6f15-4de8-9317-5f110de5838f/volumes" Jul 09 07:13:00.238809 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.238777 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:00.335707 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.335670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:00.394662 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.394628 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:00.424550 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.424517 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:13:00.424852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.424839 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e36e9b7-6f15-4de8-9317-5f110de5838f" containerName="authorino" Jul 09 07:13:00.424852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.424853 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e36e9b7-6f15-4de8-9317-5f110de5838f" containerName="authorino" Jul 09 07:13:00.424940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.424916 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e36e9b7-6f15-4de8-9317-5f110de5838f" containerName="authorino" Jul 09 07:13:00.429195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.429179 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.431922 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.431903 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mxwxw\"" Jul 09 07:13:00.432038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.431943 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 07:13:00.432038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.431979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 07:13:00.440673 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.440649 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:13:00.478689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.478659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:00.495253 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.495191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.495347 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.495253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28spj\" (UniqueName: \"kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.596198 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.596159 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.596366 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.596296 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28spj\" (UniqueName: \"kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.598583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.598560 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.605136 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.605113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28spj\" (UniqueName: \"kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj\") pod \"maas-controller-65ccf64d86-xf9j5\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.740169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.740134 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:00.872375 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:00.872345 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:13:00.875291 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:13:00.875255 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe5ecfc6_1845_400e_b2de_223c55a7024e.slice/crio-6fc37a466e0570703b1a584e9186f86afb75ffc98ab8ad6a483c5b5110d7055c WatchSource:0}: Error finding container 6fc37a466e0570703b1a584e9186f86afb75ffc98ab8ad6a483c5b5110d7055c: Status 404 returned error can't find the container with id 6fc37a466e0570703b1a584e9186f86afb75ffc98ab8ad6a483c5b5110d7055c Jul 09 07:13:01.675447 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:01.675350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" event={"ID":"be5ecfc6-1845-400e-b2de-223c55a7024e","Type":"ContainerStarted","Data":"6fc37a466e0570703b1a584e9186f86afb75ffc98ab8ad6a483c5b5110d7055c"} Jul 09 07:13:02.951617 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:02.951579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:03.043438 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:03.043331 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:03.687121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:03.687087 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" event={"ID":"be5ecfc6-1845-400e-b2de-223c55a7024e","Type":"ContainerStarted","Data":"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b"} Jul 09 07:13:03.687272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:03.687202 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:03.713437 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:03.713369 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" podStartSLOduration=1.833332611 podStartE2EDuration="3.713353336s" totalImagesPullingTime="1.880020725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:13:03.711167624 +0000 UTC m=+638.385024462" watchObservedRunningTime="2026-07-09 07:13:03.713353336 +0000 UTC m=+638.387210185" Jul 09 07:13:14.697410 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:14.697364 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:13:19.645660 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:19.645626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:35.478469 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:35.478431 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:36.061821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.061782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:36.125979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.125943 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt"] Jul 09 07:13:36.129481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.129451 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.131774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.131755 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tsnn4\"" Jul 09 07:13:36.131889 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.131866 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 07:13:36.142334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.142305 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt"] Jul 09 07:13:36.201787 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.201763 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:36.206937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.206910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd420394-41b5-49fb-b002-9fd247229fa3-maas-api-tls\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.207063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.206955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gp22\" (UniqueName: \"kubernetes.io/projected/fd420394-41b5-49fb-b002-9fd247229fa3-kube-api-access-7gp22\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.207063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.207046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd420394-41b5-49fb-b002-9fd247229fa3-openshift-service-ca\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.307840 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.307809 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd420394-41b5-49fb-b002-9fd247229fa3-openshift-service-ca\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.308026 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.307879 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd420394-41b5-49fb-b002-9fd247229fa3-maas-api-tls\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.308026 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.307909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gp22\" (UniqueName: \"kubernetes.io/projected/fd420394-41b5-49fb-b002-9fd247229fa3-kube-api-access-7gp22\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.308454 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.308433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd420394-41b5-49fb-b002-9fd247229fa3-openshift-service-ca\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.310195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.310179 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd420394-41b5-49fb-b002-9fd247229fa3-maas-api-tls\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.319283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.319232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gp22\" (UniqueName: \"kubernetes.io/projected/fd420394-41b5-49fb-b002-9fd247229fa3-kube-api-access-7gp22\") pod \"maas-api-7bf76ddcf6-7hnkt\" (UID: \"fd420394-41b5-49fb-b002-9fd247229fa3\") " pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.440892 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.440860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:36.588571 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.588491 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt"] Jul 09 07:13:36.591722 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:13:36.591687 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd420394_41b5_49fb_b002_9fd247229fa3.slice/crio-285dfaa40d1e236c0df3848499f7b584bb2466500f101bc2a0c34e8b0d7a91ab WatchSource:0}: Error finding container 285dfaa40d1e236c0df3848499f7b584bb2466500f101bc2a0c34e8b0d7a91ab: Status 404 returned error can't find the container with id 285dfaa40d1e236c0df3848499f7b584bb2466500f101bc2a0c34e8b0d7a91ab Jul 09 07:13:36.605379 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.605353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:36.699370 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.699338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:36.735913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.735874 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7sr8x"] Jul 09 07:13:36.738873 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.738856 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.741264 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.741245 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m45gv\"" Jul 09 07:13:36.741404 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.741377 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 07:13:36.750194 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.750171 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7sr8x"] Jul 09 07:13:36.811815 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.811784 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d19221d-a578-4069-b83a-0ea24f018d90-plugins-config-volume\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.811986 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.811821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndh8\" (UniqueName: \"kubernetes.io/projected/1d19221d-a578-4069-b83a-0ea24f018d90-kube-api-access-lndh8\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.851469 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.851376 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" event={"ID":"fd420394-41b5-49fb-b002-9fd247229fa3","Type":"ContainerStarted","Data":"285dfaa40d1e236c0df3848499f7b584bb2466500f101bc2a0c34e8b0d7a91ab"} Jul 09 07:13:36.912992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.912956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d19221d-a578-4069-b83a-0ea24f018d90-plugins-config-volume\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.912992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.912996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndh8\" (UniqueName: \"kubernetes.io/projected/1d19221d-a578-4069-b83a-0ea24f018d90-kube-api-access-lndh8\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.913691 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.913671 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d19221d-a578-4069-b83a-0ea24f018d90-plugins-config-volume\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:36.921225 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:36.921200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndh8\" (UniqueName: \"kubernetes.io/projected/1d19221d-a578-4069-b83a-0ea24f018d90-kube-api-access-lndh8\") pod \"payload-processing-58f86c7798-7sr8x\" (UID: \"1d19221d-a578-4069-b83a-0ea24f018d90\") " pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:37.050975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:37.050947 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:37.185127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:37.185090 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7sr8x"] Jul 09 07:13:37.186488 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:13:37.186460 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d19221d_a578_4069_b83a_0ea24f018d90.slice/crio-758f8cda70cd56cd950d7932567008b55a24c98ca1132b34ed01507818203344 WatchSource:0}: Error finding container 758f8cda70cd56cd950d7932567008b55a24c98ca1132b34ed01507818203344: Status 404 returned error can't find the container with id 758f8cda70cd56cd950d7932567008b55a24c98ca1132b34ed01507818203344 Jul 09 07:13:37.859547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:37.859509 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" event={"ID":"1d19221d-a578-4069-b83a-0ea24f018d90","Type":"ContainerStarted","Data":"758f8cda70cd56cd950d7932567008b55a24c98ca1132b34ed01507818203344"} Jul 09 07:13:39.872670 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.872626 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" event={"ID":"fd420394-41b5-49fb-b002-9fd247229fa3","Type":"ContainerStarted","Data":"1b80f45fc0f6989e56561194758b3c32c3986334ad2582c5248d5acfd6b6f125"} Jul 09 07:13:39.873168 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.872796 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:39.874323 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.874298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" event={"ID":"1d19221d-a578-4069-b83a-0ea24f018d90","Type":"ContainerStarted","Data":"61cf91f2a73ec9ad1aff62111ed62c60cb835104917aa083b1f6d0392dc777aa"} Jul 09 07:13:39.874468 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.874366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:13:39.890140 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.890095 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" podStartSLOduration=1.312140442 podStartE2EDuration="3.890083099s" totalImagesPullingTime="2.577942657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:13:39.888195562 +0000 UTC m=+674.562052410" watchObservedRunningTime="2026-07-09 07:13:39.890083099 +0000 UTC m=+674.563939949" Jul 09 07:13:39.903999 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:39.903955 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" podStartSLOduration=1.9190510189999999 podStartE2EDuration="3.903942447s" totalImagesPullingTime="1.984891428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:13:39.90204754 +0000 UTC m=+674.575904408" watchObservedRunningTime="2026-07-09 07:13:39.903942447 +0000 UTC m=+674.577799295" Jul 09 07:13:44.905433 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.905378 2573 generic.go:356] "Generic (PLEG): container finished" podID="6d1f9a59-576b-40f6-bd3a-98fcf6249a1a" containerID="410cecd64e8b03b366d5cf785c66481c54d5fd50c52216b9fed7e241b8c2ba03" exitCode=0 Jul 09 07:13:44.905849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.905454 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" event={"ID":"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a","Type":"ContainerDied","Data":"410cecd64e8b03b366d5cf785c66481c54d5fd50c52216b9fed7e241b8c2ba03"} Jul 09 07:13:44.905849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.905835 2573 scope.go:117] "RemoveContainer" containerID="410cecd64e8b03b366d5cf785c66481c54d5fd50c52216b9fed7e241b8c2ba03" Jul 09 07:13:44.906804 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.906774 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:13:44.906911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.906843 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:13:44.906911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:44.906878 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 09 07:13:45.886053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:45.886027 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bf76ddcf6-7hnkt" Jul 09 07:13:45.917183 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:45.917148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" event={"ID":"6d1f9a59-576b-40f6-bd3a-98fcf6249a1a","Type":"ContainerStarted","Data":"99150321762a69ada87511853324d765ec34856f22ee73e08ebd5383e1cd970c"} Jul 09 07:13:46.499852 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:46.499816 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:13:46.500948 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:46.500922 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mfbgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 07:13:46.501080 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:46.500980 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" podUID="6d1f9a59-576b-40f6-bd3a-98fcf6249a1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 07:13:46.917131 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:46.917085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:47.071253 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:47.071217 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:13:47.500791 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:47.500755 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mfbgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 07:13:47.500963 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:47.500812 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" podUID="6d1f9a59-576b-40f6-bd3a-98fcf6249a1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 07:13:48.500766 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:48.500728 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mfbgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 07:13:48.501202 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:48.500799 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" podUID="6d1f9a59-576b-40f6-bd3a-98fcf6249a1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 07:13:49.421224 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:49.421194 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:13:49.500891 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:49.500859 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mfbgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 07:13:49.501269 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:49.500922 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" podUID="6d1f9a59-576b-40f6-bd3a-98fcf6249a1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 07:13:50.503682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:50.503653 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:13:50.504704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:50.504686 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mfbgr" Jul 09 07:13:50.882989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:13:50.882896 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-7sr8x" Jul 09 07:14:04.736679 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:14:04.736626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:14:04.837630 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:14:04.837597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:14:13.728877 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:14:13.728845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:14:34.396301 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:14:34.396266 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:14:47.930325 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:14:47.930287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:00.136122 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.136082 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:15:00.138433 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.138415 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:15:00.147640 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.147615 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:15:00.164112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.164086 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ffb4\" (UniqueName: \"kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4\") pod \"maas-api-key-cleanup-29726355-7wtwr\" (UID: \"68bb527c-51a1-4f78-9deb-2fa4ade8038c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:15:00.265130 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.265086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ffb4\" (UniqueName: \"kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4\") pod \"maas-api-key-cleanup-29726355-7wtwr\" (UID: \"68bb527c-51a1-4f78-9deb-2fa4ade8038c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:15:00.273935 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.273905 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ffb4\" (UniqueName: \"kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4\") pod \"maas-api-key-cleanup-29726355-7wtwr\" (UID: \"68bb527c-51a1-4f78-9deb-2fa4ade8038c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:15:00.449570 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.449473 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:15:00.581024 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:00.580998 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:15:00.582617 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:15:00.582588 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68bb527c_51a1_4f78_9deb_2fa4ade8038c.slice/crio-4c6ff2063bea8c68e497d8462da1f5faffe261cffbc838bcc1695e9270c8e916 WatchSource:0}: Error finding container 4c6ff2063bea8c68e497d8462da1f5faffe261cffbc838bcc1695e9270c8e916: Status 404 returned error can't find the container with id 4c6ff2063bea8c68e497d8462da1f5faffe261cffbc838bcc1695e9270c8e916 Jul 09 07:15:01.293286 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:01.293251 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" event={"ID":"68bb527c-51a1-4f78-9deb-2fa4ade8038c","Type":"ContainerStarted","Data":"4c6ff2063bea8c68e497d8462da1f5faffe261cffbc838bcc1695e9270c8e916"} Jul 09 07:15:03.304707 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:03.304671 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" event={"ID":"68bb527c-51a1-4f78-9deb-2fa4ade8038c","Type":"ContainerStarted","Data":"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e"} Jul 09 07:15:03.320312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:03.320259 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" podStartSLOduration=1.211300211 podStartE2EDuration="3.320241567s" totalImagesPullingTime="2.108941356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:15:03.319727662 +0000 UTC m=+757.993584505" watchObservedRunningTime="2026-07-09 07:15:03.320241567 +0000 UTC m=+757.994098413" Jul 09 07:15:05.722886 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:05.722854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:11.416485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:11.416455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:17.403134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:17.403095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:50.353917 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:50.353835 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:53.859508 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:53.859467 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69d759fc69-rq7d9"] Jul 09 07:15:53.862724 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:53.862703 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:53.871070 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:53.871048 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69d759fc69-rq7d9"] Jul 09 07:15:53.948873 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:53.948842 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljf5w\" (UniqueName: \"kubernetes.io/projected/a07d1665-71c0-48a3-9de2-bcc8be4519bb-kube-api-access-ljf5w\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:53.949056 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:53.948910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a07d1665-71c0-48a3-9de2-bcc8be4519bb-tls-cert\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.050227 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.050196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a07d1665-71c0-48a3-9de2-bcc8be4519bb-tls-cert\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.050427 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.050248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljf5w\" (UniqueName: \"kubernetes.io/projected/a07d1665-71c0-48a3-9de2-bcc8be4519bb-kube-api-access-ljf5w\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.052789 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.052758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a07d1665-71c0-48a3-9de2-bcc8be4519bb-tls-cert\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.058604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.058577 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljf5w\" (UniqueName: \"kubernetes.io/projected/a07d1665-71c0-48a3-9de2-bcc8be4519bb-kube-api-access-ljf5w\") pod \"authorino-69d759fc69-rq7d9\" (UID: \"a07d1665-71c0-48a3-9de2-bcc8be4519bb\") " pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.172594 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.172559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69d759fc69-rq7d9" Jul 09 07:15:54.318095 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.318069 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69d759fc69-rq7d9"] Jul 09 07:15:54.318980 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:15:54.318951 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda07d1665_71c0_48a3_9de2_bcc8be4519bb.slice/crio-97fdd9db0c818d29ade25ce65d78258d3b99f0c0ffcf03cd09b38bf7eff51050 WatchSource:0}: Error finding container 97fdd9db0c818d29ade25ce65d78258d3b99f0c0ffcf03cd09b38bf7eff51050: Status 404 returned error can't find the container with id 97fdd9db0c818d29ade25ce65d78258d3b99f0c0ffcf03cd09b38bf7eff51050 Jul 09 07:15:54.319740 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.319719 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:54.429272 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.429189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:54.562757 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:54.562721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69d759fc69-rq7d9" event={"ID":"a07d1665-71c0-48a3-9de2-bcc8be4519bb","Type":"ContainerStarted","Data":"97fdd9db0c818d29ade25ce65d78258d3b99f0c0ffcf03cd09b38bf7eff51050"} Jul 09 07:15:55.500284 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.500221 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:55.570473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.570436 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69d759fc69-rq7d9" event={"ID":"a07d1665-71c0-48a3-9de2-bcc8be4519bb","Type":"ContainerStarted","Data":"95fea14963feea72358cd4afa50118ae3ccc4fd2792b1b4002384dc488a05cae"} Jul 09 07:15:55.581021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.580969 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:15:55.595534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.595475 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69d759fc69-rq7d9" podStartSLOduration=2.229914405 podStartE2EDuration="2.595455632s" totalImagesPullingTime="365.541227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:15:55.593024294 +0000 UTC m=+810.266881142" watchObservedRunningTime="2026-07-09 07:15:55.595455632 +0000 UTC m=+810.269312481" Jul 09 07:15:55.643423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.643213 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:15:55.647414 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.643855 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" podUID="8a25647f-d893-4f9d-bb21-a5612e7da60c" containerName="authorino" containerID="cri-o://c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7" gracePeriod=30 Jul 09 07:15:55.918681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.917728 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:15:55.966937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.966896 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert\") pod \"8a25647f-d893-4f9d-bb21-a5612e7da60c\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " Jul 09 07:15:55.967113 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.967016 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjtcv\" (UniqueName: \"kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv\") pod \"8a25647f-d893-4f9d-bb21-a5612e7da60c\" (UID: \"8a25647f-d893-4f9d-bb21-a5612e7da60c\") " Jul 09 07:15:55.969379 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.969338 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv" (OuterVolumeSpecName: "kube-api-access-wjtcv") pod "8a25647f-d893-4f9d-bb21-a5612e7da60c" (UID: "8a25647f-d893-4f9d-bb21-a5612e7da60c"). InnerVolumeSpecName "kube-api-access-wjtcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:15:55.978763 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:55.978726 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8a25647f-d893-4f9d-bb21-a5612e7da60c" (UID: "8a25647f-d893-4f9d-bb21-a5612e7da60c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:15:56.068552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.068522 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjtcv\" (UniqueName: \"kubernetes.io/projected/8a25647f-d893-4f9d-bb21-a5612e7da60c-kube-api-access-wjtcv\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:15:56.068552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.068548 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a25647f-d893-4f9d-bb21-a5612e7da60c-tls-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:15:56.578053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.578021 2573 generic.go:356] "Generic (PLEG): container finished" podID="8a25647f-d893-4f9d-bb21-a5612e7da60c" containerID="c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7" exitCode=0 Jul 09 07:15:56.578538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.578069 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" Jul 09 07:15:56.578538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.578106 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" event={"ID":"8a25647f-d893-4f9d-bb21-a5612e7da60c","Type":"ContainerDied","Data":"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7"} Jul 09 07:15:56.578538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.578160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c979fd5f6-vpfph" event={"ID":"8a25647f-d893-4f9d-bb21-a5612e7da60c","Type":"ContainerDied","Data":"b632ad06b16431ab93b3c6956e07a662dfbfcf0b37c0f9bcc468106888a93101"} Jul 09 07:15:56.578538 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.578184 2573 scope.go:117] "RemoveContainer" containerID="c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7" Jul 09 07:15:56.587667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.587652 2573 scope.go:117] "RemoveContainer" containerID="c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7" Jul 09 07:15:56.587926 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:15:56.587908 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7\": container with ID starting with c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7 not found: ID does not exist" containerID="c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7" Jul 09 07:15:56.587984 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.587935 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7"} err="failed to get container status \"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7\": rpc error: code = NotFound desc = could not find container \"c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7\": container with ID starting with c1ad8bdcf93e59be4d1256006ea039e5f7dbe2e2d43a53dc5fd4a5883f91f6c7 not found: ID does not exist" Jul 09 07:15:56.600737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.600710 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:15:56.604535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:56.604507 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c979fd5f6-vpfph"] Jul 09 07:15:57.917383 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:15:57.917346 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a25647f-d893-4f9d-bb21-a5612e7da60c" path="/var/lib/kubelet/pods/8a25647f-d893-4f9d-bb21-a5612e7da60c/volumes" Jul 09 07:16:00.014602 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:00.014569 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:16:00.015082 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:00.014826 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" podUID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" containerName="cleanup" containerID="cri-o://6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e" gracePeriod=30 Jul 09 07:16:07.702692 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:07.702657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:16:30.197118 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.197087 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:16:30.377321 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.377238 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ffb4\" (UniqueName: \"kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4\") pod \"68bb527c-51a1-4f78-9deb-2fa4ade8038c\" (UID: \"68bb527c-51a1-4f78-9deb-2fa4ade8038c\") " Jul 09 07:16:30.379327 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.379306 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4" (OuterVolumeSpecName: "kube-api-access-2ffb4") pod "68bb527c-51a1-4f78-9deb-2fa4ade8038c" (UID: "68bb527c-51a1-4f78-9deb-2fa4ade8038c"). InnerVolumeSpecName "kube-api-access-2ffb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:16:30.478246 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.478216 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ffb4\" (UniqueName: \"kubernetes.io/projected/68bb527c-51a1-4f78-9deb-2fa4ade8038c-kube-api-access-2ffb4\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:16:30.754380 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.754348 2573 generic.go:356] "Generic (PLEG): container finished" podID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" containerID="6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e" exitCode=137 Jul 09 07:16:30.754625 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.754409 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" Jul 09 07:16:30.754625 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.754443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" event={"ID":"68bb527c-51a1-4f78-9deb-2fa4ade8038c","Type":"ContainerDied","Data":"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e"} Jul 09 07:16:30.754625 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.754500 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr" event={"ID":"68bb527c-51a1-4f78-9deb-2fa4ade8038c","Type":"ContainerDied","Data":"4c6ff2063bea8c68e497d8462da1f5faffe261cffbc838bcc1695e9270c8e916"} Jul 09 07:16:30.754625 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.754527 2573 scope.go:117] "RemoveContainer" containerID="6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e" Jul 09 07:16:30.763189 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.763170 2573 scope.go:117] "RemoveContainer" containerID="6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e" Jul 09 07:16:30.763440 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:16:30.763419 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e\": container with ID starting with 6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e not found: ID does not exist" containerID="6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e" Jul 09 07:16:30.763510 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.763450 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e"} err="failed to get container status \"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e\": rpc error: code = NotFound desc = could not find container \"6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e\": container with ID starting with 6e0e1bd7cc33ffb7c73da6ca848c243cfc369c27e7b6af1ec38e77928ff4b53e not found: ID does not exist" Jul 09 07:16:30.778484 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.778459 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:16:30.781102 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:30.781080 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726355-7wtwr"] Jul 09 07:16:31.923324 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:31.923275 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" path="/var/lib/kubelet/pods/68bb527c-51a1-4f78-9deb-2fa4ade8038c/volumes" Jul 09 07:16:44.076729 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:44.076696 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:16:44.157780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:16:44.157749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:17:09.319646 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.319605 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:17:09.393743 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.393708 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:17:09.393979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.393922 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" podUID="be5ecfc6-1845-400e-b2de-223c55a7024e" containerName="manager" containerID="cri-o://6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b" gracePeriod=30 Jul 09 07:17:09.643767 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.643740 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:17:09.702159 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.702133 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert\") pod \"be5ecfc6-1845-400e-b2de-223c55a7024e\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " Jul 09 07:17:09.702306 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.702196 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28spj\" (UniqueName: \"kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj\") pod \"be5ecfc6-1845-400e-b2de-223c55a7024e\" (UID: \"be5ecfc6-1845-400e-b2de-223c55a7024e\") " Jul 09 07:17:09.704552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.704520 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj" (OuterVolumeSpecName: "kube-api-access-28spj") pod "be5ecfc6-1845-400e-b2de-223c55a7024e" (UID: "be5ecfc6-1845-400e-b2de-223c55a7024e"). InnerVolumeSpecName "kube-api-access-28spj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:17:09.704667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.704549 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "be5ecfc6-1845-400e-b2de-223c55a7024e" (UID: "be5ecfc6-1845-400e-b2de-223c55a7024e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:17:09.803238 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.803201 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28spj\" (UniqueName: \"kubernetes.io/projected/be5ecfc6-1845-400e-b2de-223c55a7024e-kube-api-access-28spj\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:17:09.803238 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.803232 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be5ecfc6-1845-400e-b2de-223c55a7024e-webhook-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:17:09.958913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.958883 2573 generic.go:356] "Generic (PLEG): container finished" podID="be5ecfc6-1845-400e-b2de-223c55a7024e" containerID="6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b" exitCode=0 Jul 09 07:17:09.959063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.958930 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" Jul 09 07:17:09.959063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.958953 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" event={"ID":"be5ecfc6-1845-400e-b2de-223c55a7024e","Type":"ContainerDied","Data":"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b"} Jul 09 07:17:09.959063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.958994 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-xf9j5" event={"ID":"be5ecfc6-1845-400e-b2de-223c55a7024e","Type":"ContainerDied","Data":"6fc37a466e0570703b1a584e9186f86afb75ffc98ab8ad6a483c5b5110d7055c"} Jul 09 07:17:09.959063 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.959016 2573 scope.go:117] "RemoveContainer" containerID="6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b" Jul 09 07:17:09.967766 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.967747 2573 scope.go:117] "RemoveContainer" containerID="6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b" Jul 09 07:17:09.968043 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:17:09.968017 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b\": container with ID starting with 6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b not found: ID does not exist" containerID="6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b" Jul 09 07:17:09.968122 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.968051 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b"} err="failed to get container status \"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b\": rpc error: code = NotFound desc = could not find container \"6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b\": container with ID starting with 6db57906d7e1b2d70b808fe8e13530c973294551dd0b5beee9597673f9f90c8b not found: ID does not exist" Jul 09 07:17:09.981096 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.981071 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:17:09.984919 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:09.984900 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-xf9j5"] Jul 09 07:17:10.797193 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.797157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:17:10.899041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899005 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65ccf64d86-6tfjx"] Jul 09 07:17:10.899427 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899408 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" containerName="cleanup" Jul 09 07:17:10.899520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899429 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" containerName="cleanup" Jul 09 07:17:10.899520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899451 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be5ecfc6-1845-400e-b2de-223c55a7024e" containerName="manager" Jul 09 07:17:10.899520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899460 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="be5ecfc6-1845-400e-b2de-223c55a7024e" containerName="manager" Jul 09 07:17:10.899520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899474 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a25647f-d893-4f9d-bb21-a5612e7da60c" containerName="authorino" Jul 09 07:17:10.899520 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899483 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a25647f-d893-4f9d-bb21-a5612e7da60c" containerName="authorino" Jul 09 07:17:10.899765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899569 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="be5ecfc6-1845-400e-b2de-223c55a7024e" containerName="manager" Jul 09 07:17:10.899765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899594 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="68bb527c-51a1-4f78-9deb-2fa4ade8038c" containerName="cleanup" Jul 09 07:17:10.899765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.899605 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a25647f-d893-4f9d-bb21-a5612e7da60c" containerName="authorino" Jul 09 07:17:10.904169 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.904143 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:10.908613 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.908590 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 07:17:10.908730 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.908590 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mxwxw\"" Jul 09 07:17:10.908730 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.908593 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 07:17:10.920893 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:10.920869 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-6tfjx"] Jul 09 07:17:11.012867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.012829 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4756ffb-aebe-4995-99fc-b41c59a5ba73-webhook-cert\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.013061 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.012881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br5xx\" (UniqueName: \"kubernetes.io/projected/a4756ffb-aebe-4995-99fc-b41c59a5ba73-kube-api-access-br5xx\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.113937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.113854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4756ffb-aebe-4995-99fc-b41c59a5ba73-webhook-cert\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.113937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.113900 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br5xx\" (UniqueName: \"kubernetes.io/projected/a4756ffb-aebe-4995-99fc-b41c59a5ba73-kube-api-access-br5xx\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.116406 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.116364 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4756ffb-aebe-4995-99fc-b41c59a5ba73-webhook-cert\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.125482 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.125451 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br5xx\" (UniqueName: \"kubernetes.io/projected/a4756ffb-aebe-4995-99fc-b41c59a5ba73-kube-api-access-br5xx\") pod \"maas-controller-65ccf64d86-6tfjx\" (UID: \"a4756ffb-aebe-4995-99fc-b41c59a5ba73\") " pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.214083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.214049 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:11.360900 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.360873 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65ccf64d86-6tfjx"] Jul 09 07:17:11.361771 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:17:11.361746 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4756ffb_aebe_4995_99fc_b41c59a5ba73.slice/crio-eb8c3de7e2f2531af5f638e6d90553335cb140d8c83813b7ebece9a7ee049412 WatchSource:0}: Error finding container eb8c3de7e2f2531af5f638e6d90553335cb140d8c83813b7ebece9a7ee049412: Status 404 returned error can't find the container with id eb8c3de7e2f2531af5f638e6d90553335cb140d8c83813b7ebece9a7ee049412 Jul 09 07:17:11.918101 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.918074 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be5ecfc6-1845-400e-b2de-223c55a7024e" path="/var/lib/kubelet/pods/be5ecfc6-1845-400e-b2de-223c55a7024e/volumes" Jul 09 07:17:11.972110 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.972069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" event={"ID":"a4756ffb-aebe-4995-99fc-b41c59a5ba73","Type":"ContainerStarted","Data":"147ab7583a21628dc75c97a9d8f1798eddbd99cfd9630b0c1fb7a177be7b6ff4"} Jul 09 07:17:11.972288 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.972128 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" event={"ID":"a4756ffb-aebe-4995-99fc-b41c59a5ba73","Type":"ContainerStarted","Data":"eb8c3de7e2f2531af5f638e6d90553335cb140d8c83813b7ebece9a7ee049412"} Jul 09 07:17:11.972288 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:11.972158 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:12.008594 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:12.008544 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" podStartSLOduration=1.4879624439999999 podStartE2EDuration="2.008528682s" totalImagesPullingTime="520.566238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:17:12.007367675 +0000 UTC m=+886.681224522" watchObservedRunningTime="2026-07-09 07:17:12.008528682 +0000 UTC m=+886.682385534" Jul 09 07:17:22.985006 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:22.984930 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65ccf64d86-6tfjx" Jul 09 07:17:25.792471 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:25.792444 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:17:25.794500 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:25.794478 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:17:52.367271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:52.367234 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:17:52.529773 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:52.529738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:17:52.660273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:17:52.660238 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:18:00.419923 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:18:00.419886 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:18:01.791314 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:18:01.791279 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:18:07.357508 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:18:07.357476 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:18:07.493204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:18:07.493169 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:18:44.189585 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:18:44.189552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:19:43.507979 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:19:43.507939 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:20:22.620757 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:20:22.620672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:20:22.721737 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:20:22.721703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:22:25.828465 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:22:25.828433 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:22:25.830442 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:22:25.830418 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:23:03.135074 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.135035 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7"] Jul 09 07:23:03.138401 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.138371 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.140929 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.140909 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gw8h4\"" Jul 09 07:23:03.152853 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.152831 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7"] Jul 09 07:23:03.220364 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.220324 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrxjd\" (UniqueName: \"kubernetes.io/projected/abf20ae1-4d09-44cf-a368-e4b4079a8499-kube-api-access-vrxjd\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.220558 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.220381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abf20ae1-4d09-44cf-a368-e4b4079a8499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.321332 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.321292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrxjd\" (UniqueName: \"kubernetes.io/projected/abf20ae1-4d09-44cf-a368-e4b4079a8499-kube-api-access-vrxjd\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.321547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.321353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abf20ae1-4d09-44cf-a368-e4b4079a8499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.321788 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.321765 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/abf20ae1-4d09-44cf-a368-e4b4079a8499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.330592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.330570 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrxjd\" (UniqueName: \"kubernetes.io/projected/abf20ae1-4d09-44cf-a368-e4b4079a8499-kube-api-access-vrxjd\") pod \"kuadrant-operator-controller-manager-86878c769f-cdcc7\" (UID: \"abf20ae1-4d09-44cf-a368-e4b4079a8499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.448304 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.448208 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.584784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.584754 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7"] Jul 09 07:23:03.585983 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:23:03.585958 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf20ae1_4d09_44cf_a368_e4b4079a8499.slice/crio-4a71d8b77db831bd17b5e1316b1b16c306102d50c705c54ce88f6c7cdc551c2e WatchSource:0}: Error finding container 4a71d8b77db831bd17b5e1316b1b16c306102d50c705c54ce88f6c7cdc551c2e: Status 404 returned error can't find the container with id 4a71d8b77db831bd17b5e1316b1b16c306102d50c705c54ce88f6c7cdc551c2e Jul 09 07:23:03.773466 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.773358 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" event={"ID":"abf20ae1-4d09-44cf-a368-e4b4079a8499","Type":"ContainerStarted","Data":"eb73ffbcd9e3f623337f9a86d73ffe48cbbc22229ea5b805264f4beee62cbeea"} Jul 09 07:23:03.773466 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.773442 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" event={"ID":"abf20ae1-4d09-44cf-a368-e4b4079a8499","Type":"ContainerStarted","Data":"4a71d8b77db831bd17b5e1316b1b16c306102d50c705c54ce88f6c7cdc551c2e"} Jul 09 07:23:03.773652 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.773470 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:23:03.794113 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:03.794057 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" podStartSLOduration=0.794038716 podStartE2EDuration="794.038716ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:23:03.793165809 +0000 UTC m=+1238.467022655" watchObservedRunningTime="2026-07-09 07:23:03.794038716 +0000 UTC m=+1238.467895564" Jul 09 07:23:05.555795 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:05.555759 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:23:07.048596 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:07.048561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:23:14.780247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:23:14.780216 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdcc7" Jul 09 07:24:07.226630 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:24:07.226588 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:24:07.325845 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:24:07.325812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:25:05.371890 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:25:05.371812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:26:21.687376 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:26:21.687294 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:27:25.860124 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:27:25.860093 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:27:25.862758 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:27:25.862737 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:28:45.350091 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:28:45.350052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:28:50.028671 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:28:50.028632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:28:58.295094 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:28:58.295050 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:03.408707 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:03.408670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:06.260369 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:06.260336 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:06.371209 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:06.371177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:06.752308 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:06.752248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:12.537021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:12.536984 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:12.635250 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:12.635209 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:12.754308 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:12.754273 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:12.856622 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:12.856542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:13.935774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:13.935746 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:14.035313 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.035280 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:14.185882 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.185794 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:14.189517 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.189495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.191665 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.191644 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-37ae4b70-serving-cert\"" Jul 09 07:29:14.201627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.201602 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:14.220154 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.220125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:14.303341 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.303309 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.303550 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.303375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmx5p\" (UniqueName: \"kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.303550 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.303493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.353910 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.353881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:14.403959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.403930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmx5p\" (UniqueName: \"kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.404170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.403978 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.404170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.404102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.404728 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.404704 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.406671 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.406649 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.412593 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.412572 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmx5p\" (UniqueName: \"kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p\") pod \"maas-api-e2e-ait-37ae4b70-586566b758-549mf\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.500562 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.500467 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:14.635813 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.635781 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:14.636973 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:29:14.636916 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98734ee7_19fe_408d_ab76_553322ebc4cd.slice/crio-cac7a4a4c76bf4c7a672e3033ba4237e4de7a14569ecb2b3945a2d455f821ad6 WatchSource:0}: Error finding container cac7a4a4c76bf4c7a672e3033ba4237e4de7a14569ecb2b3945a2d455f821ad6: Status 404 returned error can't find the container with id cac7a4a4c76bf4c7a672e3033ba4237e4de7a14569ecb2b3945a2d455f821ad6 Jul 09 07:29:14.638255 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.638225 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:29:14.707105 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.707073 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:14.756355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.756262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" event={"ID":"98734ee7-19fe-408d-ab76-553322ebc4cd","Type":"ContainerStarted","Data":"cac7a4a4c76bf4c7a672e3033ba4237e4de7a14569ecb2b3945a2d455f821ad6"} Jul 09 07:29:14.808959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:14.808927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:15.763213 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:15.763172 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" event={"ID":"98734ee7-19fe-408d-ab76-553322ebc4cd","Type":"ContainerStarted","Data":"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05"} Jul 09 07:29:15.763689 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:15.763259 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:15.780774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:15.780711 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" podStartSLOduration=1.220289267 podStartE2EDuration="1.780689369s" totalImagesPullingTime="560.400102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:29:15.778867944 +0000 UTC m=+1610.452724815" watchObservedRunningTime="2026-07-09 07:29:15.780689369 +0000 UTC m=+1610.454546217" Jul 09 07:29:16.704061 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:16.704027 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:18.947249 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:18.947210 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:19.128768 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.128727 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:19.231344 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.231254 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:19.306875 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.306842 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:19.307200 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.307170 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" podUID="98734ee7-19fe-408d-ab76-553322ebc4cd" containerName="maas-api" containerID="cri-o://ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05" gracePeriod=30 Jul 09 07:29:19.335928 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.335894 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:19.556958 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.556929 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:19.748367 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.748336 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls\") pod \"98734ee7-19fe-408d-ab76-553322ebc4cd\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " Jul 09 07:29:19.748562 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.748431 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca\") pod \"98734ee7-19fe-408d-ab76-553322ebc4cd\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " Jul 09 07:29:19.748562 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.748513 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmx5p\" (UniqueName: \"kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p\") pod \"98734ee7-19fe-408d-ab76-553322ebc4cd\" (UID: \"98734ee7-19fe-408d-ab76-553322ebc4cd\") " Jul 09 07:29:19.748762 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.748739 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98734ee7-19fe-408d-ab76-553322ebc4cd" (UID: "98734ee7-19fe-408d-ab76-553322ebc4cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:29:19.748838 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.748822 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98734ee7-19fe-408d-ab76-553322ebc4cd-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:19.750408 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.750376 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98734ee7-19fe-408d-ab76-553322ebc4cd" (UID: "98734ee7-19fe-408d-ab76-553322ebc4cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:29:19.750561 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.750544 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p" (OuterVolumeSpecName: "kube-api-access-bmx5p") pod "98734ee7-19fe-408d-ab76-553322ebc4cd" (UID: "98734ee7-19fe-408d-ab76-553322ebc4cd"). InnerVolumeSpecName "kube-api-access-bmx5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:29:19.786334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.786265 2573 generic.go:356] "Generic (PLEG): container finished" podID="98734ee7-19fe-408d-ab76-553322ebc4cd" containerID="ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05" exitCode=0 Jul 09 07:29:19.786334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.786308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" event={"ID":"98734ee7-19fe-408d-ab76-553322ebc4cd","Type":"ContainerDied","Data":"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05"} Jul 09 07:29:19.786334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.786315 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" Jul 09 07:29:19.786574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.786350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf" event={"ID":"98734ee7-19fe-408d-ab76-553322ebc4cd","Type":"ContainerDied","Data":"cac7a4a4c76bf4c7a672e3033ba4237e4de7a14569ecb2b3945a2d455f821ad6"} Jul 09 07:29:19.786574 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.786373 2573 scope.go:117] "RemoveContainer" containerID="ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05" Jul 09 07:29:19.795180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.795165 2573 scope.go:117] "RemoveContainer" containerID="ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05" Jul 09 07:29:19.795448 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:19.795419 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05\": container with ID starting with ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05 not found: ID does not exist" containerID="ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05" Jul 09 07:29:19.795534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.795456 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05"} err="failed to get container status \"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05\": rpc error: code = NotFound desc = could not find container \"ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05\": container with ID starting with ea8a694f6bcf6016ff40dfe48066ad8e3f483cc7018512e3e006292c7f279f05 not found: ID does not exist" Jul 09 07:29:19.809811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.809785 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:19.812761 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.812743 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37ae4b70-586566b758-549mf"] Jul 09 07:29:19.850247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.850219 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmx5p\" (UniqueName: \"kubernetes.io/projected/98734ee7-19fe-408d-ab76-553322ebc4cd-kube-api-access-bmx5p\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:19.850247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.850242 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98734ee7-19fe-408d-ab76-553322ebc4cd-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:19.917134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:19.917102 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98734ee7-19fe-408d-ab76-553322ebc4cd" path="/var/lib/kubelet/pods/98734ee7-19fe-408d-ab76-553322ebc4cd/volumes" Jul 09 07:29:20.644061 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:20.643998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:25.111767 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:25.111738 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:25.176712 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:25.176676 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:25.363312 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:25.363227 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:25.463738 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:25.463706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:26.540954 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.540912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:26.640015 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.639981 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:26.709317 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.709283 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:26.709720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.709704 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98734ee7-19fe-408d-ab76-553322ebc4cd" containerName="maas-api" Jul 09 07:29:26.709720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.709721 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="98734ee7-19fe-408d-ab76-553322ebc4cd" containerName="maas-api" Jul 09 07:29:26.709826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.709798 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="98734ee7-19fe-408d-ab76-553322ebc4cd" containerName="maas-api" Jul 09 07:29:26.712721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.712702 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.715810 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.715778 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-27d28868-serving-cert\"" Jul 09 07:29:26.724360 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.724334 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:26.747784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.747756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:26.803206 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.803119 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.803206 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.803163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.803430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.803292 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtqwx\" (UniqueName: \"kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.903713 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.903671 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.903713 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.903718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.903937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.903779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtqwx\" (UniqueName: \"kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.903937 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:26.903892 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-serving-cert: secret "maas-api-e2e-ait-27d28868-serving-cert" not found Jul 09 07:29:26.904022 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:26.903968 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls podName:71be79b2-bac9-4ee1-aa08-ac6d8152e902 nodeName:}" failed. No retries permitted until 2026-07-09 07:29:27.403945716 +0000 UTC m=+1622.077802557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls") pod "maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" (UID: "71be79b2-bac9-4ee1-aa08-ac6d8152e902") : secret "maas-api-e2e-ait-27d28868-serving-cert" not found Jul 09 07:29:26.904447 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.904428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:26.917510 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:26.917479 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtqwx\" (UniqueName: \"kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:27.248023 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:27.247991 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:27.350592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:27.350555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:27.407597 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:27.407559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:27.407758 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:27.407734 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-serving-cert: secret "maas-api-e2e-ait-27d28868-serving-cert" not found Jul 09 07:29:27.407840 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:27.407828 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls podName:71be79b2-bac9-4ee1-aa08-ac6d8152e902 nodeName:}" failed. No retries permitted until 2026-07-09 07:29:28.407804839 +0000 UTC m=+1623.081661665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls") pod "maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" (UID: "71be79b2-bac9-4ee1-aa08-ac6d8152e902") : secret "maas-api-e2e-ait-27d28868-serving-cert" not found Jul 09 07:29:27.492533 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:27.492502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:28.416024 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.415992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:28.418545 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.418515 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:28.460693 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.460662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:28.525039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.525006 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:28.660045 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.660016 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:28.662197 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:29:28.662166 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71be79b2_bac9_4ee1_aa08_ac6d8152e902.slice/crio-d81724abcfbc9a3a8f62508cf992bf1268b2595d5a1f2ae37d6a54674fac7acb WatchSource:0}: Error finding container d81724abcfbc9a3a8f62508cf992bf1268b2595d5a1f2ae37d6a54674fac7acb: Status 404 returned error can't find the container with id d81724abcfbc9a3a8f62508cf992bf1268b2595d5a1f2ae37d6a54674fac7acb Jul 09 07:29:28.837411 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:28.837286 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" event={"ID":"71be79b2-bac9-4ee1-aa08-ac6d8152e902","Type":"ContainerStarted","Data":"d81724abcfbc9a3a8f62508cf992bf1268b2595d5a1f2ae37d6a54674fac7acb"} Jul 09 07:29:29.844471 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:29.844420 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" event={"ID":"71be79b2-bac9-4ee1-aa08-ac6d8152e902","Type":"ContainerStarted","Data":"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b"} Jul 09 07:29:29.844829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:29.844509 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:29.860638 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:29.860582 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" podStartSLOduration=3.415952197 podStartE2EDuration="3.860565129s" totalImagesPullingTime="444.612932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:29:29.860327621 +0000 UTC m=+1624.534184497" watchObservedRunningTime="2026-07-09 07:29:29.860565129 +0000 UTC m=+1624.534421980" Jul 09 07:29:31.587671 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:31.587631 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:32.010602 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.010568 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:32.010831 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.010801 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" podUID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" containerName="maas-api" containerID="cri-o://5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b" gracePeriod=30 Jul 09 07:29:32.039058 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.039027 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:32.259887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.259863 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:32.352416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.352312 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") pod \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " Jul 09 07:29:32.352416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.352376 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtqwx\" (UniqueName: \"kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx\") pod \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " Jul 09 07:29:32.352416 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.352418 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca\") pod \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\" (UID: \"71be79b2-bac9-4ee1-aa08-ac6d8152e902\") " Jul 09 07:29:32.352826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.352803 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71be79b2-bac9-4ee1-aa08-ac6d8152e902" (UID: "71be79b2-bac9-4ee1-aa08-ac6d8152e902"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:29:32.354430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.354383 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71be79b2-bac9-4ee1-aa08-ac6d8152e902" (UID: "71be79b2-bac9-4ee1-aa08-ac6d8152e902"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:29:32.354555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.354539 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx" (OuterVolumeSpecName: "kube-api-access-jtqwx") pod "71be79b2-bac9-4ee1-aa08-ac6d8152e902" (UID: "71be79b2-bac9-4ee1-aa08-ac6d8152e902"). InnerVolumeSpecName "kube-api-access-jtqwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:29:32.453204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.453170 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71be79b2-bac9-4ee1-aa08-ac6d8152e902-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:32.453204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.453198 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtqwx\" (UniqueName: \"kubernetes.io/projected/71be79b2-bac9-4ee1-aa08-ac6d8152e902-kube-api-access-jtqwx\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:32.453204 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.453208 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71be79b2-bac9-4ee1-aa08-ac6d8152e902-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:32.862021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.861987 2573 generic.go:356] "Generic (PLEG): container finished" podID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" containerID="5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b" exitCode=0 Jul 09 07:29:32.862462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.862032 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" Jul 09 07:29:32.862462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.862073 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" event={"ID":"71be79b2-bac9-4ee1-aa08-ac6d8152e902","Type":"ContainerDied","Data":"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b"} Jul 09 07:29:32.862462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.862125 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q" event={"ID":"71be79b2-bac9-4ee1-aa08-ac6d8152e902","Type":"ContainerDied","Data":"d81724abcfbc9a3a8f62508cf992bf1268b2595d5a1f2ae37d6a54674fac7acb"} Jul 09 07:29:32.862462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.862148 2573 scope.go:117] "RemoveContainer" containerID="5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b" Jul 09 07:29:32.870844 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.870824 2573 scope.go:117] "RemoveContainer" containerID="5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b" Jul 09 07:29:32.871081 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:32.871062 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b\": container with ID starting with 5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b not found: ID does not exist" containerID="5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b" Jul 09 07:29:32.871146 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.871089 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b"} err="failed to get container status \"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b\": rpc error: code = NotFound desc = could not find container \"5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b\": container with ID starting with 5721ef89f18d4fd653a30a663b85fd38172a40cb316e310242aeacf803edb12b not found: ID does not exist" Jul 09 07:29:32.885203 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.885178 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:32.890859 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:32.890839 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-27d28868-5bbd4bd996-dsh2q"] Jul 09 07:29:33.918249 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:33.918218 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" path="/var/lib/kubelet/pods/71be79b2-bac9-4ee1-aa08-ac6d8152e902/volumes" Jul 09 07:29:37.651606 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:37.651567 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:37.761790 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:37.761756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:39.778549 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:39.778518 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:43.867380 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:43.867341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:43.966234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:43.966199 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:44.089717 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:44.089686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:44.191436 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:44.191382 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:45.196741 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.196698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:45.277791 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.277760 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:45.278148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.278135 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" containerName="maas-api" Jul 09 07:29:45.278148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.278150 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" containerName="maas-api" Jul 09 07:29:45.278249 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.278207 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="71be79b2-bac9-4ee1-aa08-ac6d8152e902" containerName="maas-api" Jul 09 07:29:45.282789 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.282772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.285373 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.285347 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b1f9470e-serving-cert\"" Jul 09 07:29:45.286148 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.286124 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:45.289924 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.289902 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:45.363256 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.363221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.363444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.363289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.363444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.363315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4696m\" (UniqueName: \"kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.370110 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.370066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:45.464657 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.464573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.464657 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.464608 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4696m\" (UniqueName: \"kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.464839 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.464666 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.465244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.465209 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.467043 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.467021 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.476095 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.476075 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4696m\" (UniqueName: \"kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m\") pod \"maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.548054 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.548020 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:45.595170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.595138 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:45.737290 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.737263 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:45.739948 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:29:45.739917 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd716e799_c2a2_44de_9fc1_d0417b734261.slice/crio-be2363a449ad64cb69a47663933b1e037296551b1d619593f4a9d2210db8765a WatchSource:0}: Error finding container be2363a449ad64cb69a47663933b1e037296551b1d619593f4a9d2210db8765a: Status 404 returned error can't find the container with id be2363a449ad64cb69a47663933b1e037296551b1d619593f4a9d2210db8765a Jul 09 07:29:45.929201 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.929168 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" event={"ID":"d716e799-c2a2-44de-9fc1-d0417b734261","Type":"ContainerStarted","Data":"be2363a449ad64cb69a47663933b1e037296551b1d619593f4a9d2210db8765a"} Jul 09 07:29:45.971927 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:45.971897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:46.078304 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:46.078217 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:46.937734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:46.937694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" event={"ID":"d716e799-c2a2-44de-9fc1-d0417b734261","Type":"ContainerStarted","Data":"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f"} Jul 09 07:29:46.938185 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:46.937771 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:46.956308 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:46.956257 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" podStartSLOduration=1.384288881 podStartE2EDuration="1.956240483s" totalImagesPullingTime="571.951602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:29:46.954469476 +0000 UTC m=+1641.628326325" watchObservedRunningTime="2026-07-09 07:29:46.956240483 +0000 UTC m=+1641.630097330" Jul 09 07:29:47.302606 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:47.302577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:49.807897 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:49.807862 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:49.952547 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:49.952503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:50.051596 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.051559 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:50.191153 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.191114 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:50.191368 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.191319 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" podUID="d716e799-c2a2-44de-9fc1-d0417b734261" containerName="maas-api" containerID="cri-o://1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f" gracePeriod=30 Jul 09 07:29:50.224145 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.224110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:50.448566 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.448507 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:50.512053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.512020 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls\") pod \"d716e799-c2a2-44de-9fc1-d0417b734261\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " Jul 09 07:29:50.512053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.512053 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4696m\" (UniqueName: \"kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m\") pod \"d716e799-c2a2-44de-9fc1-d0417b734261\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " Jul 09 07:29:50.512302 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.512117 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca\") pod \"d716e799-c2a2-44de-9fc1-d0417b734261\" (UID: \"d716e799-c2a2-44de-9fc1-d0417b734261\") " Jul 09 07:29:50.512523 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.512494 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d716e799-c2a2-44de-9fc1-d0417b734261" (UID: "d716e799-c2a2-44de-9fc1-d0417b734261"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:29:50.514205 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.514183 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d716e799-c2a2-44de-9fc1-d0417b734261" (UID: "d716e799-c2a2-44de-9fc1-d0417b734261"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:29:50.514288 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.514190 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m" (OuterVolumeSpecName: "kube-api-access-4696m") pod "d716e799-c2a2-44de-9fc1-d0417b734261" (UID: "d716e799-c2a2-44de-9fc1-d0417b734261"). InnerVolumeSpecName "kube-api-access-4696m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:29:50.612697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.612659 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d716e799-c2a2-44de-9fc1-d0417b734261-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:50.612697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.612689 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4696m\" (UniqueName: \"kubernetes.io/projected/d716e799-c2a2-44de-9fc1-d0417b734261-kube-api-access-4696m\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:50.612697 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.612700 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d716e799-c2a2-44de-9fc1-d0417b734261-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:29:50.961038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.961000 2573 generic.go:356] "Generic (PLEG): container finished" podID="d716e799-c2a2-44de-9fc1-d0417b734261" containerID="1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f" exitCode=0 Jul 09 07:29:50.961492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.961043 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" Jul 09 07:29:50.961492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.961083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" event={"ID":"d716e799-c2a2-44de-9fc1-d0417b734261","Type":"ContainerDied","Data":"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f"} Jul 09 07:29:50.961492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.961135 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp" event={"ID":"d716e799-c2a2-44de-9fc1-d0417b734261","Type":"ContainerDied","Data":"be2363a449ad64cb69a47663933b1e037296551b1d619593f4a9d2210db8765a"} Jul 09 07:29:50.961492 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.961159 2573 scope.go:117] "RemoveContainer" containerID="1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f" Jul 09 07:29:50.971152 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.971134 2573 scope.go:117] "RemoveContainer" containerID="1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f" Jul 09 07:29:50.971445 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:29:50.971425 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f\": container with ID starting with 1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f not found: ID does not exist" containerID="1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f" Jul 09 07:29:50.971531 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.971455 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f"} err="failed to get container status \"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f\": rpc error: code = NotFound desc = could not find container \"1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f\": container with ID starting with 1cbd71c0e74a9b9545140ff493a7d7059733532c30dbb99ee2f2ccea9677722f not found: ID does not exist" Jul 09 07:29:50.984407 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.984363 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:50.987036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:50.987010 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b1f9470e-f75d4bb5d-8sxmp"] Jul 09 07:29:51.917681 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:51.917650 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d716e799-c2a2-44de-9fc1-d0417b734261" path="/var/lib/kubelet/pods/d716e799-c2a2-44de-9fc1-d0417b734261/volumes" Jul 09 07:29:51.958754 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:51.958726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:55.980452 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:55.980411 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:56.042953 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:56.042918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:56.477535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:56.477489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:56.582122 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:56.582076 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:56.698992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:56.698960 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:29:58.052128 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:29:58.052097 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:03.974823 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:03.974788 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:04.075196 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:04.075161 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:04.175814 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:04.175782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:04.277005 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:04.276918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:04.689569 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:04.689530 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:30.653143 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:30.653089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:31.042822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:31.042721 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:31.143474 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:31.143444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:36.713072 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:36.713038 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:36.788449 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:36.788385 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:36.864205 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:36.864168 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:36.962694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:36.962665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:52.838636 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:52.838548 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:58.800033 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:58.800000 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:58.902220 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:58.902172 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:30:59.000582 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:30:59.000553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:09.511881 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:09.511844 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:10.328519 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:10.328482 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:16.071109 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:16.071071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:16.141830 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:16.141800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:16.261799 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:16.261754 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:16.415772 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:16.415741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:17.258014 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:17.257979 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:17.336607 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:17.336569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:19.003953 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:19.003920 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:24.889998 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:24.889951 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:25.013273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:25.013238 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:25.586134 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:25.586101 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:31.432066 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:31.432032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:31.498081 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:31.498050 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:31.680447 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:31.680388 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:31.994911 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:31.994879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:32.515380 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:32.515349 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:34.785486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.785444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:34.857207 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.857177 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:34.857666 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.857648 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d716e799-c2a2-44de-9fc1-d0417b734261" containerName="maas-api" Jul 09 07:31:34.857741 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.857668 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d716e799-c2a2-44de-9fc1-d0417b734261" containerName="maas-api" Jul 09 07:31:34.857741 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.857726 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d716e799-c2a2-44de-9fc1-d0417b734261" containerName="maas-api" Jul 09 07:31:34.860733 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.860716 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:34.863575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.863557 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0c3ae94c-serving-cert\"" Jul 09 07:31:34.871910 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.871886 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:34.877975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.877929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:34.901823 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.901802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:34.901950 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.901844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:34.901950 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.901910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dltc\" (UniqueName: \"kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:34.944542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:34.944512 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:35.003122 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.003091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.003323 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.003154 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.003323 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.003205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dltc\" (UniqueName: \"kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.003924 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.003895 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.005718 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.005689 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.018124 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.018100 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dltc\" (UniqueName: \"kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc\") pod \"maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.059362 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.059262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:35.171510 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.171480 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:35.306945 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.306921 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:35.308431 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:31:35.308387 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17639065_a590_4258_89ce_10b2b8825e0e.slice/crio-7845757dc7d626eefbe67854315912c546d5fe3fdc5ca14607812e7660c480bb WatchSource:0}: Error finding container 7845757dc7d626eefbe67854315912c546d5fe3fdc5ca14607812e7660c480bb: Status 404 returned error can't find the container with id 7845757dc7d626eefbe67854315912c546d5fe3fdc5ca14607812e7660c480bb Jul 09 07:31:35.434064 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.434032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:35.493624 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.493591 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" event={"ID":"17639065-a590-4258-89ce-10b2b8825e0e","Type":"ContainerStarted","Data":"7845757dc7d626eefbe67854315912c546d5fe3fdc5ca14607812e7660c480bb"} Jul 09 07:31:35.534676 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:35.534641 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:36.500595 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:36.500556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" event={"ID":"17639065-a590-4258-89ce-10b2b8825e0e","Type":"ContainerStarted","Data":"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810"} Jul 09 07:31:36.501066 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:36.500635 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:36.518078 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:36.518025 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" podStartSLOduration=2.11328344 podStartE2EDuration="2.518008785s" totalImagesPullingTime="404.725345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:31:36.516995001 +0000 UTC m=+1751.190851871" watchObservedRunningTime="2026-07-09 07:31:36.518008785 +0000 UTC m=+1751.191865632" Jul 09 07:31:39.289720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:39.289684 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:39.953873 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:39.953843 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:40.055143 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:40.055110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:40.434816 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:40.434778 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:42.511914 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:42.511887 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:46.103604 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.103572 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:46.203840 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.203807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:46.516807 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.516776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:46.899212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.899182 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:46.899448 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.899420 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" podUID="17639065-a590-4258-89ce-10b2b8825e0e" containerName="maas-api" containerID="cri-o://6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810" gracePeriod=30 Jul 09 07:31:46.912240 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.912198 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:46.971341 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:46.971315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:47.143249 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.143224 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:47.208137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.208059 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls\") pod \"17639065-a590-4258-89ce-10b2b8825e0e\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " Jul 09 07:31:47.208137 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.208118 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dltc\" (UniqueName: \"kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc\") pod \"17639065-a590-4258-89ce-10b2b8825e0e\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " Jul 09 07:31:47.208324 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.208295 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca\") pod \"17639065-a590-4258-89ce-10b2b8825e0e\" (UID: \"17639065-a590-4258-89ce-10b2b8825e0e\") " Jul 09 07:31:47.208658 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.208634 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17639065-a590-4258-89ce-10b2b8825e0e" (UID: "17639065-a590-4258-89ce-10b2b8825e0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:31:47.210159 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.210133 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17639065-a590-4258-89ce-10b2b8825e0e" (UID: "17639065-a590-4258-89ce-10b2b8825e0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:31:47.210237 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.210167 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc" (OuterVolumeSpecName: "kube-api-access-9dltc") pod "17639065-a590-4258-89ce-10b2b8825e0e" (UID: "17639065-a590-4258-89ce-10b2b8825e0e"). InnerVolumeSpecName "kube-api-access-9dltc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:31:47.309806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.309770 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dltc\" (UniqueName: \"kubernetes.io/projected/17639065-a590-4258-89ce-10b2b8825e0e-kube-api-access-9dltc\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:31:47.309806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.309800 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17639065-a590-4258-89ce-10b2b8825e0e-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:31:47.309806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.309812 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17639065-a590-4258-89ce-10b2b8825e0e-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:31:47.567165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.567080 2573 generic.go:356] "Generic (PLEG): container finished" podID="17639065-a590-4258-89ce-10b2b8825e0e" containerID="6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810" exitCode=0 Jul 09 07:31:47.567165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.567121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" event={"ID":"17639065-a590-4258-89ce-10b2b8825e0e","Type":"ContainerDied","Data":"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810"} Jul 09 07:31:47.567165 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.567137 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" Jul 09 07:31:47.567428 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.567183 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx" event={"ID":"17639065-a590-4258-89ce-10b2b8825e0e","Type":"ContainerDied","Data":"7845757dc7d626eefbe67854315912c546d5fe3fdc5ca14607812e7660c480bb"} Jul 09 07:31:47.567428 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.567206 2573 scope.go:117] "RemoveContainer" containerID="6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810" Jul 09 07:31:47.577229 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.577211 2573 scope.go:117] "RemoveContainer" containerID="6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810" Jul 09 07:31:47.577552 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:31:47.577517 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810\": container with ID starting with 6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810 not found: ID does not exist" containerID="6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810" Jul 09 07:31:47.577656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.577558 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810"} err="failed to get container status \"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810\": rpc error: code = NotFound desc = could not find container \"6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810\": container with ID starting with 6c42dad9da615a6159f9befc006e8c3a5fb1cea44a33298a2778123508500810 not found: ID does not exist" Jul 09 07:31:47.588462 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.588433 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:47.593806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.593782 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c3ae94c-84689d77c6-wrctx"] Jul 09 07:31:47.917414 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:47.917362 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17639065-a590-4258-89ce-10b2b8825e0e" path="/var/lib/kubelet/pods/17639065-a590-4258-89ce-10b2b8825e0e/volumes" Jul 09 07:31:52.651504 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:52.651470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:52.751243 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:52.751211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:52.936647 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:52.936531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:53.280989 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:53.280892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:53.411026 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:53.410993 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:53.524593 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:53.524561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:59.271856 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:59.271823 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:59.384317 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:59.384280 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:59.474151 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:59.474112 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:31:59.615720 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:31:59.615632 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:05.678956 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:05.678922 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:05.742657 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:05.742626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:12.507937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:12.507898 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:12.839767 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:12.839686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:12.970103 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:12.970071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:15.980660 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:15.980623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.075780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.075744 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.200434 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.200381 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:16.200980 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.200959 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17639065-a590-4258-89ce-10b2b8825e0e" containerName="maas-api" Jul 09 07:32:16.201036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.200985 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="17639065-a590-4258-89ce-10b2b8825e0e" containerName="maas-api" Jul 09 07:32:16.201100 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.201087 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="17639065-a590-4258-89ce-10b2b8825e0e" containerName="maas-api" Jul 09 07:32:16.204402 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.204375 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.206484 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.206465 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b0405232-serving-cert\"" Jul 09 07:32:16.214671 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.214650 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:16.236584 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.236509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.365608 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.365572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.365774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.365652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzlgt\" (UniqueName: \"kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.365774 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.365697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.466771 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.466742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzlgt\" (UniqueName: \"kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.466940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.466789 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.466940 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.466839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.467019 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:32:16.466951 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-serving-cert: secret "maas-api-e2e-mt-b0405232-serving-cert" not found Jul 09 07:32:16.467019 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:32:16.467017 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls podName:395907cd-db43-482e-b4b1-13bbe9eddf36 nodeName:}" failed. No retries permitted until 2026-07-09 07:32:16.966997081 +0000 UTC m=+1791.640853913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls") pod "maas-api-e2e-mt-b0405232-69f8f8984b-glms6" (UID: "395907cd-db43-482e-b4b1-13bbe9eddf36") : secret "maas-api-e2e-mt-b0405232-serving-cert" not found Jul 09 07:32:16.467490 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.467465 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.475282 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.475254 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzlgt\" (UniqueName: \"kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.665003 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.664970 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.764843 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.764813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.864705 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.864668 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:16.971544 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.971464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:16.973906 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:16.973886 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") pod \"maas-api-e2e-mt-b0405232-69f8f8984b-glms6\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:17.115938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:17.115870 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:17.265590 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:17.265562 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:17.267950 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:32:17.267922 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395907cd_db43_482e_b4b1_13bbe9eddf36.slice/crio-9b287e1871a371c747dee8a01ffd5a0ae2b90e5a57c9c782cb6770b4276f53a0 WatchSource:0}: Error finding container 9b287e1871a371c747dee8a01ffd5a0ae2b90e5a57c9c782cb6770b4276f53a0: Status 404 returned error can't find the container with id 9b287e1871a371c747dee8a01ffd5a0ae2b90e5a57c9c782cb6770b4276f53a0 Jul 09 07:32:17.728970 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:17.728936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" event={"ID":"395907cd-db43-482e-b4b1-13bbe9eddf36","Type":"ContainerStarted","Data":"9b287e1871a371c747dee8a01ffd5a0ae2b90e5a57c9c782cb6770b4276f53a0"} Jul 09 07:32:18.737195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:18.737160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" event={"ID":"395907cd-db43-482e-b4b1-13bbe9eddf36","Type":"ContainerStarted","Data":"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3"} Jul 09 07:32:18.737659 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:18.737237 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:18.753090 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:18.752995 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" podStartSLOduration=1.373657581 podStartE2EDuration="2.752977201s" totalImagesPullingTime="1.37931962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:32:18.752773871 +0000 UTC m=+1793.426630727" watchObservedRunningTime="2026-07-09 07:32:18.752977201 +0000 UTC m=+1793.426834050" Jul 09 07:32:20.460533 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:20.460500 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:24.749026 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:24.748999 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:25.893792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:25.893761 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:32:25.897606 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:25.897580 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:32:34.155522 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:34.155488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:34.249115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:34.249080 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:37.438849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:37.438820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:37.540040 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:37.539997 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:37.829968 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:37.829885 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:58.328198 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.328163 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:58.328688 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.328493 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" podUID="395907cd-db43-482e-b4b1-13bbe9eddf36" containerName="maas-api" containerID="cri-o://0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3" gracePeriod=30 Jul 09 07:32:58.574195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.574171 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:58.751294 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.751260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca\") pod \"395907cd-db43-482e-b4b1-13bbe9eddf36\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " Jul 09 07:32:58.751294 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.751300 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzlgt\" (UniqueName: \"kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt\") pod \"395907cd-db43-482e-b4b1-13bbe9eddf36\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " Jul 09 07:32:58.751537 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.751353 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") pod \"395907cd-db43-482e-b4b1-13bbe9eddf36\" (UID: \"395907cd-db43-482e-b4b1-13bbe9eddf36\") " Jul 09 07:32:58.751667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.751646 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "395907cd-db43-482e-b4b1-13bbe9eddf36" (UID: "395907cd-db43-482e-b4b1-13bbe9eddf36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:32:58.753631 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.753606 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "395907cd-db43-482e-b4b1-13bbe9eddf36" (UID: "395907cd-db43-482e-b4b1-13bbe9eddf36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:32:58.753750 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.753652 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt" (OuterVolumeSpecName: "kube-api-access-bzlgt") pod "395907cd-db43-482e-b4b1-13bbe9eddf36" (UID: "395907cd-db43-482e-b4b1-13bbe9eddf36"). InnerVolumeSpecName "kube-api-access-bzlgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:32:58.852315 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.852285 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395907cd-db43-482e-b4b1-13bbe9eddf36-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:32:58.852315 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.852317 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395907cd-db43-482e-b4b1-13bbe9eddf36-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:32:58.852542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.852327 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzlgt\" (UniqueName: \"kubernetes.io/projected/395907cd-db43-482e-b4b1-13bbe9eddf36-kube-api-access-bzlgt\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:32:58.852542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.852287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:58.948075 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.948041 2573 generic.go:356] "Generic (PLEG): container finished" podID="395907cd-db43-482e-b4b1-13bbe9eddf36" containerID="0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3" exitCode=0 Jul 09 07:32:58.948223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.948087 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" Jul 09 07:32:58.948223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.948125 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" event={"ID":"395907cd-db43-482e-b4b1-13bbe9eddf36","Type":"ContainerDied","Data":"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3"} Jul 09 07:32:58.948223 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.948195 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6" event={"ID":"395907cd-db43-482e-b4b1-13bbe9eddf36","Type":"ContainerDied","Data":"9b287e1871a371c747dee8a01ffd5a0ae2b90e5a57c9c782cb6770b4276f53a0"} Jul 09 07:32:58.948347 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.948238 2573 scope.go:117] "RemoveContainer" containerID="0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3" Jul 09 07:32:58.953534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.953512 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:32:58.959473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.959457 2573 scope.go:117] "RemoveContainer" containerID="0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3" Jul 09 07:32:58.959699 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:32:58.959681 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3\": container with ID starting with 0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3 not found: ID does not exist" containerID="0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3" Jul 09 07:32:58.959763 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.959705 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3"} err="failed to get container status \"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3\": rpc error: code = NotFound desc = could not find container \"0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3\": container with ID starting with 0b0f6eee273b580b541dbf88a2f0d6d6da95904ee7ad7ac55fc2f1532a4c98f3 not found: ID does not exist" Jul 09 07:32:58.973566 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.973530 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:58.977168 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:58.977148 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b0405232-69f8f8984b-glms6"] Jul 09 07:32:59.917727 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:32:59.917692 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395907cd-db43-482e-b4b1-13bbe9eddf36" path="/var/lib/kubelet/pods/395907cd-db43-482e-b4b1-13bbe9eddf36/volumes" Jul 09 07:33:04.048536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:04.048500 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:09.064566 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:09.064530 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:40.821239 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:40.821206 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:41.445355 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:41.445324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:42.987810 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:42.987779 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:43.088742 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:43.088702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:47.284582 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:47.284494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:49.675071 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:49.675036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:50.011746 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:50.011663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:50.374020 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:50.373941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:50.590827 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:50.590793 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:51.634992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:51.634955 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:58.046756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:58.046721 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:58.214652 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:58.214620 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:33:58.728563 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:33:58.728533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:05.071304 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:05.071271 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:05.174960 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:05.174925 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:05.716938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:05.716903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:15.830548 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:15.830513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:16.176758 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:16.176728 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:37.142694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:37.142662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:44.291447 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:44.291384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:44.417893 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:44.417860 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:44.659450 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:44.659413 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:45.066854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:45.066772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:45.369561 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:45.369475 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:47.732345 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.732310 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:47.816731 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.816698 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:34:47.817093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.817072 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395907cd-db43-482e-b4b1-13bbe9eddf36" containerName="maas-api" Jul 09 07:34:47.817093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.817086 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="395907cd-db43-482e-b4b1-13bbe9eddf36" containerName="maas-api" Jul 09 07:34:47.817180 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.817167 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="395907cd-db43-482e-b4b1-13bbe9eddf36" containerName="maas-api" Jul 09 07:34:47.820238 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.820222 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:47.822612 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.822587 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9c5220-serving-cert\"" Jul 09 07:34:47.832756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.832736 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:34:47.832936 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.832910 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:47.974892 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:47.974863 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:48.006641 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.006561 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.006641 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.006606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-774wm\" (UniqueName: \"kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.006870 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.006730 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.107682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.107647 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.107682 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.107686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-774wm\" (UniqueName: \"kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.107910 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.107734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.107910 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:34:48.107788 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-serving-cert: secret "maas-api-e2e-inf-a-9c5220-serving-cert" not found Jul 09 07:34:48.107910 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:34:48.107860 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls podName:ee431826-ef6b-47f8-872a-4843d846a22d nodeName:}" failed. No retries permitted until 2026-07-09 07:34:48.607844551 +0000 UTC m=+1943.281701376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls") pod "maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" (UID: "ee431826-ef6b-47f8-872a-4843d846a22d") : secret "maas-api-e2e-inf-a-9c5220-serving-cert" not found Jul 09 07:34:48.108368 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.108351 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.120156 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.120123 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-774wm\" (UniqueName: \"kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.457778 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.457741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:48.556775 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.556735 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:48.613208 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.613133 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.616014 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.615990 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.731901 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.731807 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:48.900925 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:34:48.900898 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee431826_ef6b_47f8_872a_4843d846a22d.slice/crio-c10b429ba793763de621d75338937767c645c1178bbac3325afd566bcca872fb WatchSource:0}: Error finding container c10b429ba793763de621d75338937767c645c1178bbac3325afd566bcca872fb: Status 404 returned error can't find the container with id c10b429ba793763de621d75338937767c645c1178bbac3325afd566bcca872fb Jul 09 07:34:48.902247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.902226 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:34:48.904248 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:48.904224 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:34:49.512220 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:49.512189 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" event={"ID":"ee431826-ef6b-47f8-872a-4843d846a22d","Type":"ContainerStarted","Data":"26eff0fa0f360a55d42513e728520afc48c639de251d49d38e4431075f13e9ab"} Jul 09 07:34:49.512423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:49.512241 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" event={"ID":"ee431826-ef6b-47f8-872a-4843d846a22d","Type":"ContainerStarted","Data":"c10b429ba793763de621d75338937767c645c1178bbac3325afd566bcca872fb"} Jul 09 07:34:49.512423 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:49.512353 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:49.535295 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:49.535236 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" podStartSLOduration=2.056103349 podStartE2EDuration="2.535214683s" totalImagesPullingTime="479.111334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:34:49.533853443 +0000 UTC m=+1944.207710315" watchObservedRunningTime="2026-07-09 07:34:49.535214683 +0000 UTC m=+1944.209071531" Jul 09 07:34:52.221127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:52.221089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:52.555334 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:52.555242 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:52.885428 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:52.885377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:52.985275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:52.985237 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:55.522282 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.522248 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:34:55.790444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.790338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:55.901794 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.901757 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:34:55.902721 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.902690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:55.905576 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.905560 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:55.908575 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.908554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-055963-serving-cert\"" Jul 09 07:34:55.940170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.940141 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:34:55.978182 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.978153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k7zd\" (UniqueName: \"kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:55.978337 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.978190 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:55.978337 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:55.978216 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.045310 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.045199 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:56.078711 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.078679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k7zd\" (UniqueName: \"kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.078711 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.078712 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.078950 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.078738 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.078950 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:34:56.078896 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-serving-cert: secret "maas-api-e2e-inf-b-055963-serving-cert" not found Jul 09 07:34:56.079058 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:34:56.078964 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls podName:6e94e80c-a832-49ec-81ce-19b18fa00504 nodeName:}" failed. No retries permitted until 2026-07-09 07:34:56.5789446 +0000 UTC m=+1951.252801425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls") pod "maas-api-e2e-inf-b-055963-796f66d985-2thc2" (UID: "6e94e80c-a832-49ec-81ce-19b18fa00504") : secret "maas-api-e2e-inf-b-055963-serving-cert" not found Jul 09 07:34:56.079438 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.079419 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.088493 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.088467 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k7zd\" (UniqueName: \"kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.221298 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.221265 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:56.583383 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.583347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.585806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.585785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") pod \"maas-api-e2e-inf-b-055963-796f66d985-2thc2\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.678952 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.678905 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:56.815131 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.815098 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:56.868333 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.868034 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:34:56.982795 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:56.982768 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:34:56.984206 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:34:56.984181 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e94e80c_a832_49ec_81ce_19b18fa00504.slice/crio-f9345c1bdfe0ebdb9094572f8de01a149f9e63365bca6bd0ee91241a207e1b24 WatchSource:0}: Error finding container f9345c1bdfe0ebdb9094572f8de01a149f9e63365bca6bd0ee91241a207e1b24: Status 404 returned error can't find the container with id f9345c1bdfe0ebdb9094572f8de01a149f9e63365bca6bd0ee91241a207e1b24 Jul 09 07:34:57.556473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:57.556437 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" event={"ID":"6e94e80c-a832-49ec-81ce-19b18fa00504","Type":"ContainerStarted","Data":"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a"} Jul 09 07:34:57.556704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:57.556496 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" event={"ID":"6e94e80c-a832-49ec-81ce-19b18fa00504","Type":"ContainerStarted","Data":"f9345c1bdfe0ebdb9094572f8de01a149f9e63365bca6bd0ee91241a207e1b24"} Jul 09 07:34:57.556704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:57.556608 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:34:57.581510 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:57.581463 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" podStartSLOduration=2.158426553 podStartE2EDuration="2.581444856s" totalImagesPullingTime="423.018303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:34:57.579161998 +0000 UTC m=+1952.253018867" watchObservedRunningTime="2026-07-09 07:34:57.581444856 +0000 UTC m=+1952.255301703" Jul 09 07:34:59.321915 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:34:59.321879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:00.271878 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:00.271846 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:03.566700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:03.566674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:35:10.136984 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:10.136937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:10.654011 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:10.653976 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:10.776140 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:10.776106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:17.733656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:17.733579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:17.833583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:17.833548 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:36.024126 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:36.024093 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:53.800036 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:53.799998 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:54.673593 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:54.673560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:54.841467 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:54.841370 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:55.275542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.275506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:55.456946 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.456909 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:55.556414 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.556321 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:55.722265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.722229 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:35:55.722586 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.722544 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" podUID="ee431826-ef6b-47f8-872a-4843d846a22d" containerName="maas-api" containerID="cri-o://26eff0fa0f360a55d42513e728520afc48c639de251d49d38e4431075f13e9ab" gracePeriod=30 Jul 09 07:35:55.760155 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.760127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:35:55.864552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.864517 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee431826-ef6b-47f8-872a-4843d846a22d" containerID="26eff0fa0f360a55d42513e728520afc48c639de251d49d38e4431075f13e9ab" exitCode=0 Jul 09 07:35:55.864909 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.864587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" event={"ID":"ee431826-ef6b-47f8-872a-4843d846a22d","Type":"ContainerDied","Data":"26eff0fa0f360a55d42513e728520afc48c639de251d49d38e4431075f13e9ab"} Jul 09 07:35:55.973485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:55.973463 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:35:56.028439 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.028380 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca\") pod \"ee431826-ef6b-47f8-872a-4843d846a22d\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " Jul 09 07:35:56.028600 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.028456 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") pod \"ee431826-ef6b-47f8-872a-4843d846a22d\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " Jul 09 07:35:56.028600 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.028520 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-774wm\" (UniqueName: \"kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm\") pod \"ee431826-ef6b-47f8-872a-4843d846a22d\" (UID: \"ee431826-ef6b-47f8-872a-4843d846a22d\") " Jul 09 07:35:56.028780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.028758 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee431826-ef6b-47f8-872a-4843d846a22d" (UID: "ee431826-ef6b-47f8-872a-4843d846a22d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:35:56.030564 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.030532 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee431826-ef6b-47f8-872a-4843d846a22d" (UID: "ee431826-ef6b-47f8-872a-4843d846a22d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:35:56.030685 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.030654 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm" (OuterVolumeSpecName: "kube-api-access-774wm") pod "ee431826-ef6b-47f8-872a-4843d846a22d" (UID: "ee431826-ef6b-47f8-872a-4843d846a22d"). InnerVolumeSpecName "kube-api-access-774wm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:35:56.130083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.129987 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee431826-ef6b-47f8-872a-4843d846a22d-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:35:56.130083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.130024 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee431826-ef6b-47f8-872a-4843d846a22d-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:35:56.130083 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.130037 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-774wm\" (UniqueName: \"kubernetes.io/projected/ee431826-ef6b-47f8-872a-4843d846a22d-kube-api-access-774wm\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:35:56.871093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.871064 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" Jul 09 07:35:56.871093 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.871075 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg" event={"ID":"ee431826-ef6b-47f8-872a-4843d846a22d","Type":"ContainerDied","Data":"c10b429ba793763de621d75338937767c645c1178bbac3325afd566bcca872fb"} Jul 09 07:35:56.871618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.871141 2573 scope.go:117] "RemoveContainer" containerID="26eff0fa0f360a55d42513e728520afc48c639de251d49d38e4431075f13e9ab" Jul 09 07:35:56.893946 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.893911 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:35:56.900592 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:56.900565 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c5220-664f6b5f57-t4hhg"] Jul 09 07:35:57.917867 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:35:57.917830 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee431826-ef6b-47f8-872a-4843d846a22d" path="/var/lib/kubelet/pods/ee431826-ef6b-47f8-872a-4843d846a22d/volumes" Jul 09 07:36:00.442627 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:00.442597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:37.616809 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:37.616772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:37.911029 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:37.910988 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:38.339313 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:38.339208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:38.859975 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:38.859936 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:38.971792 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:38.971756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:39.077153 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.077120 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:39.247973 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.247934 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:36:39.248470 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.248440 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" podUID="6e94e80c-a832-49ec-81ce-19b18fa00504" containerName="maas-api" containerID="cri-o://6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a" gracePeriod=30 Jul 09 07:36:39.284714 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.284690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:39.498534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.498466 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:36:39.620633 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.620586 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k7zd\" (UniqueName: \"kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd\") pod \"6e94e80c-a832-49ec-81ce-19b18fa00504\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " Jul 09 07:36:39.620811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.620689 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") pod \"6e94e80c-a832-49ec-81ce-19b18fa00504\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " Jul 09 07:36:39.620811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.620752 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca\") pod \"6e94e80c-a832-49ec-81ce-19b18fa00504\" (UID: \"6e94e80c-a832-49ec-81ce-19b18fa00504\") " Jul 09 07:36:39.621146 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.621112 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e94e80c-a832-49ec-81ce-19b18fa00504" (UID: "6e94e80c-a832-49ec-81ce-19b18fa00504"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:36:39.622772 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.622744 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e94e80c-a832-49ec-81ce-19b18fa00504" (UID: "6e94e80c-a832-49ec-81ce-19b18fa00504"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:36:39.622883 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.622784 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd" (OuterVolumeSpecName: "kube-api-access-6k7zd") pod "6e94e80c-a832-49ec-81ce-19b18fa00504" (UID: "6e94e80c-a832-49ec-81ce-19b18fa00504"). InnerVolumeSpecName "kube-api-access-6k7zd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:36:39.721832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.721783 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e94e80c-a832-49ec-81ce-19b18fa00504-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:36:39.721832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.721830 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k7zd\" (UniqueName: \"kubernetes.io/projected/6e94e80c-a832-49ec-81ce-19b18fa00504-kube-api-access-6k7zd\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:36:39.721832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:39.721840 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e94e80c-a832-49ec-81ce-19b18fa00504-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:36:40.098965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.098929 2573 generic.go:356] "Generic (PLEG): container finished" podID="6e94e80c-a832-49ec-81ce-19b18fa00504" containerID="6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a" exitCode=0 Jul 09 07:36:40.099363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.098974 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" Jul 09 07:36:40.099363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.099008 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" event={"ID":"6e94e80c-a832-49ec-81ce-19b18fa00504","Type":"ContainerDied","Data":"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a"} Jul 09 07:36:40.099363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.099058 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2" event={"ID":"6e94e80c-a832-49ec-81ce-19b18fa00504","Type":"ContainerDied","Data":"f9345c1bdfe0ebdb9094572f8de01a149f9e63365bca6bd0ee91241a207e1b24"} Jul 09 07:36:40.099363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.099081 2573 scope.go:117] "RemoveContainer" containerID="6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a" Jul 09 07:36:40.108039 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.108020 2573 scope.go:117] "RemoveContainer" containerID="6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a" Jul 09 07:36:40.108307 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:36:40.108287 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a\": container with ID starting with 6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a not found: ID does not exist" containerID="6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a" Jul 09 07:36:40.108351 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.108316 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a"} err="failed to get container status \"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a\": rpc error: code = NotFound desc = could not find container \"6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a\": container with ID starting with 6f2e851e33b6cb7cb657a64f5f1f50f376b034cbe40c2adaa7046fa57e83ce2a not found: ID does not exist" Jul 09 07:36:40.115812 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.115783 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:36:40.120271 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:40.120241 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-055963-796f66d985-2thc2"] Jul 09 07:36:41.918839 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:41.918804 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e94e80c-a832-49ec-81ce-19b18fa00504" path="/var/lib/kubelet/pods/6e94e80c-a832-49ec-81ce-19b18fa00504/volumes" Jul 09 07:36:42.654372 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:42.654335 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:42.753776 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:42.753739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:44.774992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:44.774961 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:44.846214 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:44.846181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:45.068428 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:45.068327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:45.631040 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:45.631010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:46.696710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:46.696681 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:46.886240 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:46.886208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:49.264450 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.264414 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:49.333826 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.333785 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:36:49.334307 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334290 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e94e80c-a832-49ec-81ce-19b18fa00504" containerName="maas-api" Jul 09 07:36:49.334387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334309 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e94e80c-a832-49ec-81ce-19b18fa00504" containerName="maas-api" Jul 09 07:36:49.334387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334351 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee431826-ef6b-47f8-872a-4843d846a22d" containerName="maas-api" Jul 09 07:36:49.334387 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334359 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee431826-ef6b-47f8-872a-4843d846a22d" containerName="maas-api" Jul 09 07:36:49.334578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334455 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e94e80c-a832-49ec-81ce-19b18fa00504" containerName="maas-api" Jul 09 07:36:49.334578 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.334471 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee431826-ef6b-47f8-872a-4843d846a22d" containerName="maas-api" Jul 09 07:36:49.339053 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.339025 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.342089 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.342068 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-42f976-serving-cert\"" Jul 09 07:36:49.353488 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.353466 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:36:49.373005 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.372980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:49.416656 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.416623 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gvnk\" (UniqueName: \"kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.416821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.416692 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.416821 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.416773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.469619 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.469582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:49.517494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.517382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.517494 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.517472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gvnk\" (UniqueName: \"kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.517694 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:36:49.517536 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-serving-cert: secret "maas-api-e2e-api-a-42f976-serving-cert" not found Jul 09 07:36:49.517694 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.517561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.517694 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:36:49.517598 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls podName:30f8a8f8-5666-45a0-8f2d-f5aaa37a594e nodeName:}" failed. No retries permitted until 2026-07-09 07:36:50.017581267 +0000 UTC m=+2064.691438091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls") pod "maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" (UID: "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e") : secret "maas-api-e2e-api-a-42f976-serving-cert" not found Jul 09 07:36:49.518157 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.518140 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.534865 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.534837 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gvnk\" (UniqueName: \"kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:49.724485 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:49.724454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:50.020945 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.020904 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:50.023367 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.023338 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") pod \"maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:50.025539 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.025502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:50.125552 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.125506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:50.249484 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.249451 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:50.392343 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:50.392313 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:36:50.394159 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:36:50.394130 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f8a8f8_5666_45a0_8f2d_f5aaa37a594e.slice/crio-da8fa91a82c13232fb0325842284af149a1e201c02d2cc037427cdca3e767b6a WatchSource:0}: Error finding container da8fa91a82c13232fb0325842284af149a1e201c02d2cc037427cdca3e767b6a: Status 404 returned error can't find the container with id da8fa91a82c13232fb0325842284af149a1e201c02d2cc037427cdca3e767b6a Jul 09 07:36:51.156321 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:51.156286 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" event={"ID":"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e","Type":"ContainerStarted","Data":"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af"} Jul 09 07:36:51.156481 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:51.156346 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" event={"ID":"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e","Type":"ContainerStarted","Data":"da8fa91a82c13232fb0325842284af149a1e201c02d2cc037427cdca3e767b6a"} Jul 09 07:36:51.156549 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:51.156491 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:51.174756 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:51.174629 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" podStartSLOduration=1.651237759 podStartE2EDuration="2.17461075s" totalImagesPullingTime="523.372991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:36:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:36:51.17435012 +0000 UTC m=+2065.848207011" watchObservedRunningTime="2026-07-09 07:36:51.17461075 +0000 UTC m=+2065.848467599" Jul 09 07:36:53.740123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:53.740086 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:54.193728 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:54.193694 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:54.533709 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:54.533625 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:55.005667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:55.005634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:57.167441 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.167386 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:36:57.327403 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.327350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:57.407073 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.407030 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:36:57.410185 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.410156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:57.411144 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.411119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.413856 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.413833 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ab11a1-serving-cert\"" Jul 09 07:36:57.424880 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.424853 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:36:57.485431 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.485358 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:57.593097 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.593066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.593275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.593107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqt6q\" (UniqueName: \"kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.593275 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.593180 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.694777 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.694694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.694777 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.694744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqt6q\" (UniqueName: \"kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.694977 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.694791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.695536 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.695510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.697486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.697451 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.706028 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.706006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqt6q\" (UniqueName: \"kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q\") pod \"maas-api-e2e-api-b-ab11a1-8646679865-rqkbc\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.725242 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.725217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:57.766189 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.766130 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:57.877555 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:57.877515 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:36:57.880821 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:36:57.880795 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7943548f_2297_4650_9cee_8b8ef8f25f99.slice/crio-639d0a379dc6ccf332c868ed261626ceeeaae392760ce292498d9c8fb70ed9c5 WatchSource:0}: Error finding container 639d0a379dc6ccf332c868ed261626ceeeaae392760ce292498d9c8fb70ed9c5: Status 404 returned error can't find the container with id 639d0a379dc6ccf332c868ed261626ceeeaae392760ce292498d9c8fb70ed9c5 Jul 09 07:36:58.031938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:58.031811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:58.124380 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:58.124342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:36:58.197330 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:58.197293 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" event={"ID":"7943548f-2297-4650-9cee-8b8ef8f25f99","Type":"ContainerStarted","Data":"639d0a379dc6ccf332c868ed261626ceeeaae392760ce292498d9c8fb70ed9c5"} Jul 09 07:36:59.207780 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:59.207746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" event={"ID":"7943548f-2297-4650-9cee-8b8ef8f25f99","Type":"ContainerStarted","Data":"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22"} Jul 09 07:36:59.208140 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:59.207827 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:36:59.227706 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:36:59.227650 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" podStartSLOduration=1.869003178 podStartE2EDuration="2.227631224s" totalImagesPullingTime="358.628046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:36:59.227064023 +0000 UTC m=+2073.900920881" watchObservedRunningTime="2026-07-09 07:36:59.227631224 +0000 UTC m=+2073.901488072" Jul 09 07:37:01.849822 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:01.849789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:05.218593 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:05.218563 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:37:09.952446 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:09.952410 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:10.351109 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.351017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:10.457941 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.457893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:10.554958 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.554919 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:10.723117 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.723088 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:37:10.723318 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.723296 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" podUID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" containerName="maas-api" containerID="cri-o://a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af" gracePeriod=30 Jul 09 07:37:10.750747 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.750714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:10.974965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:10.974905 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:37:11.010748 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.010719 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gvnk\" (UniqueName: \"kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk\") pod \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " Jul 09 07:37:11.010937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.010781 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca\") pod \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " Jul 09 07:37:11.010937 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.010812 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") pod \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\" (UID: \"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e\") " Jul 09 07:37:11.011213 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.011186 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" (UID: "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:37:11.012862 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.012833 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk" (OuterVolumeSpecName: "kube-api-access-8gvnk") pod "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" (UID: "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e"). InnerVolumeSpecName "kube-api-access-8gvnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:37:11.012952 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.012894 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" (UID: "30f8a8f8-5666-45a0-8f2d-f5aaa37a594e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:37:11.111543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.111499 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gvnk\" (UniqueName: \"kubernetes.io/projected/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-kube-api-access-8gvnk\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:11.111543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.111534 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:11.111543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.111543 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:11.272614 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.272526 2573 generic.go:356] "Generic (PLEG): container finished" podID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" containerID="a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af" exitCode=0 Jul 09 07:37:11.272614 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.272573 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" Jul 09 07:37:11.272853 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.272607 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" event={"ID":"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e","Type":"ContainerDied","Data":"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af"} Jul 09 07:37:11.272853 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.272670 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25" event={"ID":"30f8a8f8-5666-45a0-8f2d-f5aaa37a594e","Type":"ContainerDied","Data":"da8fa91a82c13232fb0325842284af149a1e201c02d2cc037427cdca3e767b6a"} Jul 09 07:37:11.272853 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.272700 2573 scope.go:117] "RemoveContainer" containerID="a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af" Jul 09 07:37:11.282589 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.282572 2573 scope.go:117] "RemoveContainer" containerID="a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af" Jul 09 07:37:11.282873 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:37:11.282856 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af\": container with ID starting with a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af not found: ID does not exist" containerID="a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af" Jul 09 07:37:11.282924 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.282881 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af"} err="failed to get container status \"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af\": rpc error: code = NotFound desc = could not find container \"a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af\": container with ID starting with a68fe54fdedb0aa5252968845a9bc0fe44da3edccb14cfacbb8909bf926a32af not found: ID does not exist" Jul 09 07:37:11.295804 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.295781 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:37:11.303887 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.303866 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-42f976-58bd7bf5c9-nvg25"] Jul 09 07:37:11.918195 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:11.918164 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" path="/var/lib/kubelet/pods/30f8a8f8-5666-45a0-8f2d-f5aaa37a594e/volumes" Jul 09 07:37:16.343854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:16.343816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:16.413995 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:16.413957 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:16.626132 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:16.626039 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.012295 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.012247 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.398244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.398213 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.500665 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.500633 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.599601 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.599569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.720715 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.720636 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:37:17.720913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.720881 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" podUID="7943548f-2297-4650-9cee-8b8ef8f25f99" containerName="maas-api" containerID="cri-o://37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22" gracePeriod=30 Jul 09 07:37:17.750623 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.750599 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:17.972891 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:17.972836 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:37:18.074543 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.074516 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqt6q\" (UniqueName: \"kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q\") pod \"7943548f-2297-4650-9cee-8b8ef8f25f99\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " Jul 09 07:37:18.074722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.074557 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls\") pod \"7943548f-2297-4650-9cee-8b8ef8f25f99\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " Jul 09 07:37:18.074722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.074607 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca\") pod \"7943548f-2297-4650-9cee-8b8ef8f25f99\" (UID: \"7943548f-2297-4650-9cee-8b8ef8f25f99\") " Jul 09 07:37:18.074991 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.074969 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7943548f-2297-4650-9cee-8b8ef8f25f99" (UID: "7943548f-2297-4650-9cee-8b8ef8f25f99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:37:18.076704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.076674 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q" (OuterVolumeSpecName: "kube-api-access-zqt6q") pod "7943548f-2297-4650-9cee-8b8ef8f25f99" (UID: "7943548f-2297-4650-9cee-8b8ef8f25f99"). InnerVolumeSpecName "kube-api-access-zqt6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:37:18.076801 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.076715 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7943548f-2297-4650-9cee-8b8ef8f25f99" (UID: "7943548f-2297-4650-9cee-8b8ef8f25f99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:37:18.175854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.175816 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7943548f-2297-4650-9cee-8b8ef8f25f99-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:18.175854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.175848 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqt6q\" (UniqueName: \"kubernetes.io/projected/7943548f-2297-4650-9cee-8b8ef8f25f99-kube-api-access-zqt6q\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:18.175854 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.175858 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7943548f-2297-4650-9cee-8b8ef8f25f99-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:37:18.311184 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.311103 2573 generic.go:356] "Generic (PLEG): container finished" podID="7943548f-2297-4650-9cee-8b8ef8f25f99" containerID="37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22" exitCode=0 Jul 09 07:37:18.311184 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.311146 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" Jul 09 07:37:18.311357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.311184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" event={"ID":"7943548f-2297-4650-9cee-8b8ef8f25f99","Type":"ContainerDied","Data":"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22"} Jul 09 07:37:18.311357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.311237 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc" event={"ID":"7943548f-2297-4650-9cee-8b8ef8f25f99","Type":"ContainerDied","Data":"639d0a379dc6ccf332c868ed261626ceeeaae392760ce292498d9c8fb70ed9c5"} Jul 09 07:37:18.311357 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.311263 2573 scope.go:117] "RemoveContainer" containerID="37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22" Jul 09 07:37:18.320066 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.320044 2573 scope.go:117] "RemoveContainer" containerID="37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22" Jul 09 07:37:18.320310 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:37:18.320291 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22\": container with ID starting with 37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22 not found: ID does not exist" containerID="37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22" Jul 09 07:37:18.320377 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.320321 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22"} err="failed to get container status \"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22\": rpc error: code = NotFound desc = could not find container \"37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22\": container with ID starting with 37d8840b531d98cc74de285c303e5941854f112e9cad053b3334b855e4050e22 not found: ID does not exist" Jul 09 07:37:18.331920 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.331894 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:37:18.336706 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:18.336683 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ab11a1-8646679865-rqkbc"] Jul 09 07:37:19.918101 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:19.918065 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7943548f-2297-4650-9cee-8b8ef8f25f99" path="/var/lib/kubelet/pods/7943548f-2297-4650-9cee-8b8ef8f25f99/volumes" Jul 09 07:37:23.313335 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:23.313303 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:23.386367 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:23.386334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:23.582170 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:23.582091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:24.174303 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:24.174265 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:24.508336 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:24.508249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:24.743566 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:24.743537 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:25.934618 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:25.934591 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:37:25.938711 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:25.938686 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lzjwl_4ef49bae-e1df-44b4-a2f0-aa9050189388/ovn-acl-logging/0.log" Jul 09 07:37:27.471675 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.471641 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:27.570908 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.570875 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:27.700959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.700926 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:37:27.701276 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701264 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7943548f-2297-4650-9cee-8b8ef8f25f99" containerName="maas-api" Jul 09 07:37:27.701319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701277 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7943548f-2297-4650-9cee-8b8ef8f25f99" containerName="maas-api" Jul 09 07:37:27.701319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701288 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" containerName="maas-api" Jul 09 07:37:27.701319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701293 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" containerName="maas-api" Jul 09 07:37:27.701430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701368 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f8a8f8-5666-45a0-8f2d-f5aaa37a594e" containerName="maas-api" Jul 09 07:37:27.701430 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.701377 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7943548f-2297-4650-9cee-8b8ef8f25f99" containerName="maas-api" Jul 09 07:37:27.705742 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.705720 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.710374 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.710353 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-62f792-serving-cert\"" Jul 09 07:37:27.724786 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.724731 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:37:27.761267 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.761240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-444hq\" (UniqueName: \"kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.761471 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.761281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.761471 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.761311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.793583 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.793542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:27.861765 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.861722 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.861966 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.861784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.861966 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:37:27.861875 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-serving-cert: secret "maas-api-e2e-shared-a-62f792-serving-cert" not found Jul 09 07:37:27.861966 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.861893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-444hq\" (UniqueName: \"kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.861966 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:37:27.861933 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls podName:9935ae21-e875-484c-a0ee-e1840d6b334c nodeName:}" failed. No retries permitted until 2026-07-09 07:37:28.361917658 +0000 UTC m=+2103.035774486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls") pod "maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" (UID: "9935ae21-e875-484c-a0ee-e1840d6b334c") : secret "maas-api-e2e-shared-a-62f792-serving-cert" not found Jul 09 07:37:27.862524 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.862507 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:27.890043 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:27.890009 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-444hq\" (UniqueName: \"kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:28.127976 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.127881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:28.367328 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.367296 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:28.369753 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.369726 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") pod \"maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:28.386226 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.386195 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:28.486647 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.486607 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:28.617959 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.617919 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:28.760599 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:28.760574 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:37:28.762576 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:37:28.762550 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9935ae21_e875_484c_a0ee_e1840d6b334c.slice/crio-d696118e093435facf48c5a6f2cd1dcbc14daf606d7de24d59c7c62b2cfedbe6 WatchSource:0}: Error finding container d696118e093435facf48c5a6f2cd1dcbc14daf606d7de24d59c7c62b2cfedbe6: Status 404 returned error can't find the container with id d696118e093435facf48c5a6f2cd1dcbc14daf606d7de24d59c7c62b2cfedbe6 Jul 09 07:37:29.370961 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:29.370926 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" event={"ID":"9935ae21-e875-484c-a0ee-e1840d6b334c","Type":"ContainerStarted","Data":"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd"} Jul 09 07:37:29.371132 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:29.370985 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" event={"ID":"9935ae21-e875-484c-a0ee-e1840d6b334c","Type":"ContainerStarted","Data":"d696118e093435facf48c5a6f2cd1dcbc14daf606d7de24d59c7c62b2cfedbe6"} Jul 09 07:37:29.371132 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:29.371044 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:29.393747 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:29.393698 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" podStartSLOduration=1.94077234 podStartE2EDuration="2.39368254s" totalImagesPullingTime="452.9102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:37:29.392531611 +0000 UTC m=+2104.066388474" watchObservedRunningTime="2026-07-09 07:37:29.39368254 +0000 UTC m=+2104.067539386" Jul 09 07:37:31.963734 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:31.963696 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:32.288759 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:32.288644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:32.623259 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:32.623147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:32.812212 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:32.812172 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:35.096521 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.096485 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:35.252501 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.252470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:35.387112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.387087 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:37:35.593105 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.593051 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:35.721912 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.721835 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:37:35.725127 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.725110 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.727829 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.727808 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-da4680-serving-cert\"" Jul 09 07:37:35.750045 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.750002 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:35.751921 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.751892 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:37:35.832247 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.832213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.832441 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.832265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.832441 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.832320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j67wq\" (UniqueName: \"kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.932936 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.932902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.933128 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.932958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.933128 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.932999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j67wq\" (UniqueName: \"kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.933646 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.933620 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.935524 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.935492 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.944984 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.944960 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j67wq\" (UniqueName: \"kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq\") pod \"maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:35.952933 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:35.952896 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:36.035509 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.035375 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:36.047062 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.047000 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:36.185833 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.185799 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:37:36.186604 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:37:36.186581 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103b0647_2c3b_4f2a_9877_089f9e6f3d07.slice/crio-9b4fa9e7aaaea57b3d41697691408f18bf442b9df81e0ea3ba94b83295a1e975 WatchSource:0}: Error finding container 9b4fa9e7aaaea57b3d41697691408f18bf442b9df81e0ea3ba94b83295a1e975: Status 404 returned error can't find the container with id 9b4fa9e7aaaea57b3d41697691408f18bf442b9df81e0ea3ba94b83295a1e975 Jul 09 07:37:36.344115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.344023 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:36.413473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.413433 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" event={"ID":"103b0647-2c3b-4f2a-9877-089f9e6f3d07","Type":"ContainerStarted","Data":"9b4fa9e7aaaea57b3d41697691408f18bf442b9df81e0ea3ba94b83295a1e975"} Jul 09 07:37:36.444316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:36.444268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:37.420377 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:37.420341 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" event={"ID":"103b0647-2c3b-4f2a-9877-089f9e6f3d07","Type":"ContainerStarted","Data":"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54"} Jul 09 07:37:37.420798 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:37.420465 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:37.439244 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:37.439194 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" podStartSLOduration=2.098734205 podStartE2EDuration="2.439179934s" totalImagesPullingTime="340.445729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:37:37.436894871 +0000 UTC m=+2112.110751744" watchObservedRunningTime="2026-07-09 07:37:37.439179934 +0000 UTC m=+2112.113036780" Jul 09 07:37:39.983348 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:39.983316 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:43.431327 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:43.431297 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:37:49.288363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:49.288326 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:49.389474 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:49.389433 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:58.175044 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:58.174997 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:37:58.275541 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:37:58.275512 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:07.353230 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:07.353138 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:08.979011 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:08.978980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:44.957776 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:44.957735 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:45.388146 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.388112 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:45.490486 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.490443 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:45.601571 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.601531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:45.801653 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.801564 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:40:45.802123 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.802077 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" podUID="9935ae21-e875-484c-a0ee-e1840d6b334c" containerName="maas-api" containerID="cri-o://4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd" gracePeriod=30 Jul 09 07:40:45.835120 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:45.835069 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:46.063645 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.063586 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:40:46.105160 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.105125 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") pod \"9935ae21-e875-484c-a0ee-e1840d6b334c\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " Jul 09 07:40:46.105346 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.105172 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-444hq\" (UniqueName: \"kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq\") pod \"9935ae21-e875-484c-a0ee-e1840d6b334c\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " Jul 09 07:40:46.105346 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.105304 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca\") pod \"9935ae21-e875-484c-a0ee-e1840d6b334c\" (UID: \"9935ae21-e875-484c-a0ee-e1840d6b334c\") " Jul 09 07:40:46.105699 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.105672 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9935ae21-e875-484c-a0ee-e1840d6b334c" (UID: "9935ae21-e875-484c-a0ee-e1840d6b334c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:40:46.107234 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.107211 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9935ae21-e875-484c-a0ee-e1840d6b334c" (UID: "9935ae21-e875-484c-a0ee-e1840d6b334c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:40:46.107315 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.107271 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq" (OuterVolumeSpecName: "kube-api-access-444hq") pod "9935ae21-e875-484c-a0ee-e1840d6b334c" (UID: "9935ae21-e875-484c-a0ee-e1840d6b334c"). InnerVolumeSpecName "kube-api-access-444hq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:40:46.206087 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.206037 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935ae21-e875-484c-a0ee-e1840d6b334c-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:46.206087 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.206082 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-444hq\" (UniqueName: \"kubernetes.io/projected/9935ae21-e875-484c-a0ee-e1840d6b334c-kube-api-access-444hq\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:46.206087 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.206097 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935ae21-e875-484c-a0ee-e1840d6b334c-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:46.408506 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.408472 2573 generic.go:356] "Generic (PLEG): container finished" podID="9935ae21-e875-484c-a0ee-e1840d6b334c" containerID="4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd" exitCode=0 Jul 09 07:40:46.408700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.408516 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" Jul 09 07:40:46.408700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.408565 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" event={"ID":"9935ae21-e875-484c-a0ee-e1840d6b334c","Type":"ContainerDied","Data":"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd"} Jul 09 07:40:46.408700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.408626 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk" event={"ID":"9935ae21-e875-484c-a0ee-e1840d6b334c","Type":"ContainerDied","Data":"d696118e093435facf48c5a6f2cd1dcbc14daf606d7de24d59c7c62b2cfedbe6"} Jul 09 07:40:46.408700 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.408655 2573 scope.go:117] "RemoveContainer" containerID="4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd" Jul 09 07:40:46.417950 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.417930 2573 scope.go:117] "RemoveContainer" containerID="4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd" Jul 09 07:40:46.418203 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:40:46.418185 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd\": container with ID starting with 4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd not found: ID does not exist" containerID="4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd" Jul 09 07:40:46.418265 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.418215 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd"} err="failed to get container status \"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd\": rpc error: code = NotFound desc = could not find container \"4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd\": container with ID starting with 4560bf8599435da44d1a7046eca9400b060c478fe85ba22827d745abec7435cd not found: ID does not exist" Jul 09 07:40:46.431457 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.431425 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:40:46.435704 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:46.435680 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-62f792-59b85cfdd8-vwzxk"] Jul 09 07:40:47.917969 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:47.917930 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9935ae21-e875-484c-a0ee-e1840d6b334c" path="/var/lib/kubelet/pods/9935ae21-e875-484c-a0ee-e1840d6b334c/volumes" Jul 09 07:40:51.312218 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:51.312181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:51.412470 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:51.412438 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:51.602661 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:51.602578 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.001343 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.001315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.405710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.405671 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.508820 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.508785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.602305 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.602272 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.711710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.711625 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:40:52.711965 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.711929 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" podUID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" containerName="maas-api" containerID="cri-o://1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54" gracePeriod=30 Jul 09 07:40:52.742813 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.742786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:52.969294 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:52.969232 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:40:53.067296 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.067263 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j67wq\" (UniqueName: \"kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq\") pod \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " Jul 09 07:40:53.067534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.067329 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls\") pod \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " Jul 09 07:40:53.067534 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.067441 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca\") pod \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\" (UID: \"103b0647-2c3b-4f2a-9877-089f9e6f3d07\") " Jul 09 07:40:53.067790 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.067767 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "103b0647-2c3b-4f2a-9877-089f9e6f3d07" (UID: "103b0647-2c3b-4f2a-9877-089f9e6f3d07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 07:40:53.069491 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.069466 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq" (OuterVolumeSpecName: "kube-api-access-j67wq") pod "103b0647-2c3b-4f2a-9877-089f9e6f3d07" (UID: "103b0647-2c3b-4f2a-9877-089f9e6f3d07"). InnerVolumeSpecName "kube-api-access-j67wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 07:40:53.069615 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.069576 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "103b0647-2c3b-4f2a-9877-089f9e6f3d07" (UID: "103b0647-2c3b-4f2a-9877-089f9e6f3d07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 07:40:53.168263 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.168223 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/103b0647-2c3b-4f2a-9877-089f9e6f3d07-openshift-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:53.168263 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.168257 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j67wq\" (UniqueName: \"kubernetes.io/projected/103b0647-2c3b-4f2a-9877-089f9e6f3d07-kube-api-access-j67wq\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:53.168263 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.168271 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/103b0647-2c3b-4f2a-9877-089f9e6f3d07-maas-api-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 09 07:40:53.448055 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.448023 2573 generic.go:356] "Generic (PLEG): container finished" podID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" containerID="1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54" exitCode=0 Jul 09 07:40:53.448444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.448073 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" Jul 09 07:40:53.448444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.448109 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" event={"ID":"103b0647-2c3b-4f2a-9877-089f9e6f3d07","Type":"ContainerDied","Data":"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54"} Jul 09 07:40:53.448444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.448169 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5" event={"ID":"103b0647-2c3b-4f2a-9877-089f9e6f3d07","Type":"ContainerDied","Data":"9b4fa9e7aaaea57b3d41697691408f18bf442b9df81e0ea3ba94b83295a1e975"} Jul 09 07:40:53.448444 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.448199 2573 scope.go:117] "RemoveContainer" containerID="1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54" Jul 09 07:40:53.459637 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.459621 2573 scope.go:117] "RemoveContainer" containerID="1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54" Jul 09 07:40:53.459909 ip-10-0-143-42 kubenswrapper[2573]: E0709 07:40:53.459883 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54\": container with ID starting with 1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54 not found: ID does not exist" containerID="1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54" Jul 09 07:40:53.459964 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.459922 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54"} err="failed to get container status \"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54\": rpc error: code = NotFound desc = could not find container \"1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54\": container with ID starting with 1f20d97d5aa931e6628bb1e4a3b0f8aaf7b7c5c484132d541c15bac730898b54 not found: ID does not exist" Jul 09 07:40:53.471074 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.471043 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:40:53.476041 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.476018 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da4680-786458f8f4-8tzd5"] Jul 09 07:40:53.917938 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:53.917903 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" path="/var/lib/kubelet/pods/103b0647-2c3b-4f2a-9877-089f9e6f3d07/volumes" Jul 09 07:40:58.322786 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:58.322752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:58.424069 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:58.424032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:58.635023 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:58.633509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:40:59.372295 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:40:59.372258 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69d759fc69-rq7d9_a07d1665-71c0-48a3-9de2-bcc8be4519bb/authorino/0.log" Jul 09 07:41:04.269438 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:04.269384 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65ccf64d86-6tfjx_a4756ffb-aebe-4995-99fc-b41c59a5ba73/manager/0.log" Jul 09 07:41:04.626781 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:04.626703 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c58b47-bz7pz_d22468e8-ccc7-4d4a-a46e-a77d4186320e/copy-manifests/0.log" Jul 09 07:41:04.636798 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:04.636756 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c58b47-bz7pz_d22468e8-ccc7-4d4a-a46e-a77d4186320e/manager/0.log" Jul 09 07:41:05.696934 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.696901 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/util/0.log" Jul 09 07:41:05.704207 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.704176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/pull/0.log" Jul 09 07:41:05.711144 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.711118 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/extract/0.log" Jul 09 07:41:05.825664 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.825630 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/util/0.log" Jul 09 07:41:05.832764 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.832739 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/pull/0.log" Jul 09 07:41:05.844980 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.844954 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/extract/0.log" Jul 09 07:41:05.957803 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.957719 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/util/0.log" Jul 09 07:41:05.965040 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.965016 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/pull/0.log" Jul 09 07:41:05.971731 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:05.971705 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/extract/0.log" Jul 09 07:41:06.087642 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.087615 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/pull/0.log" Jul 09 07:41:06.094866 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.094841 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/extract/0.log" Jul 09 07:41:06.101537 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.101509 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/util/0.log" Jul 09 07:41:06.226849 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.226751 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69d759fc69-rq7d9_a07d1665-71c0-48a3-9de2-bcc8be4519bb/authorino/0.log" Jul 09 07:41:06.352595 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.352562 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7mvq4_5675ae79-b653-4e49-80ce-c63881408de4/manager/0.log" Jul 09 07:41:06.464668 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.464641 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nz2hn_13b6ef49-736f-4d25-821a-2d547bbb4047/manager/0.log" Jul 09 07:41:06.705667 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.705634 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dkkxx_1243bfae-0fd7-4ee7-8450-b48a0aacc0cf/registry-server/0.log" Jul 09 07:41:06.829283 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:06.829252 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cdcc7_abf20ae1-4d09-44cf-a368-e4b4079a8499/manager/0.log" Jul 09 07:41:07.424542 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:07.424510 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf457qpr_36885451-6534-4ed7-9ebe-c95229740495/istio-proxy/0.log" Jul 09 07:41:07.912714 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:07.912682 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/1.log" Jul 09 07:41:08.167243 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:08.167158 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7sr8x_1d19221d-a578-4069-b83a-0ea24f018d90/payload-processing/0.log" Jul 09 07:41:13.198913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.198876 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:41:13.255649 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.255614 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s54zf/must-gather-sxzr6"] Jul 09 07:41:13.255960 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.255947 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9935ae21-e875-484c-a0ee-e1840d6b334c" containerName="maas-api" Jul 09 07:41:13.256013 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.255961 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9935ae21-e875-484c-a0ee-e1840d6b334c" containerName="maas-api" Jul 09 07:41:13.256013 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.255982 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" containerName="maas-api" Jul 09 07:41:13.256013 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.255988 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" containerName="maas-api" Jul 09 07:41:13.256115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.256038 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="103b0647-2c3b-4f2a-9877-089f9e6f3d07" containerName="maas-api" Jul 09 07:41:13.256115 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.256044 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9935ae21-e875-484c-a0ee-e1840d6b334c" containerName="maas-api" Jul 09 07:41:13.259197 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.259180 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.261811 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.261785 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s54zf\"/\"openshift-service-ca.crt\"" Jul 09 07:41:13.261970 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.261877 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s54zf\"/\"kube-root-ca.crt\"" Jul 09 07:41:13.262764 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.262747 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s54zf\"/\"default-dockercfg-tqqpq\"" Jul 09 07:41:13.268606 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.268585 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s54zf/must-gather-sxzr6"] Jul 09 07:41:13.279109 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.279082 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" limit=1024 Jul 09 07:41:13.346145 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.346107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47g59\" (UniqueName: \"kubernetes.io/projected/fdeaa671-9da7-4e28-93de-819cf4498a79-kube-api-access-47g59\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.346319 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.346160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdeaa671-9da7-4e28-93de-819cf4498a79-must-gather-output\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.447052 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.447013 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdeaa671-9da7-4e28-93de-819cf4498a79-must-gather-output\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.447225 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.447117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47g59\" (UniqueName: \"kubernetes.io/projected/fdeaa671-9da7-4e28-93de-819cf4498a79-kube-api-access-47g59\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.447419 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.447375 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdeaa671-9da7-4e28-93de-819cf4498a79-must-gather-output\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.455363 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.455302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47g59\" (UniqueName: \"kubernetes.io/projected/fdeaa671-9da7-4e28-93de-819cf4498a79-kube-api-access-47g59\") pod \"must-gather-sxzr6\" (UID: \"fdeaa671-9da7-4e28-93de-819cf4498a79\") " pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.569379 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.569352 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s54zf/must-gather-sxzr6" Jul 09 07:41:13.702251 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.702220 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s54zf/must-gather-sxzr6"] Jul 09 07:41:13.703593 ip-10-0-143-42 kubenswrapper[2573]: W0709 07:41:13.703567 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdeaa671_9da7_4e28_93de_819cf4498a79.slice/crio-9b2de047d4ad89527eeb5b054d6d110f535b82aa5cdfbd0ca1ee3ce93de12f55 WatchSource:0}: Error finding container 9b2de047d4ad89527eeb5b054d6d110f535b82aa5cdfbd0ca1ee3ce93de12f55: Status 404 returned error can't find the container with id 9b2de047d4ad89527eeb5b054d6d110f535b82aa5cdfbd0ca1ee3ce93de12f55 Jul 09 07:41:13.705605 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:13.705567 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 07:41:14.560021 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:14.559980 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s54zf/must-gather-sxzr6" event={"ID":"fdeaa671-9da7-4e28-93de-819cf4498a79","Type":"ContainerStarted","Data":"9b2de047d4ad89527eeb5b054d6d110f535b82aa5cdfbd0ca1ee3ce93de12f55"} Jul 09 07:41:15.570478 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:15.570438 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s54zf/must-gather-sxzr6" event={"ID":"fdeaa671-9da7-4e28-93de-819cf4498a79","Type":"ContainerStarted","Data":"cb9a8bc69b0186241cac55ef00bf721ba38b1b6540585ad7066b0f8c4fb0f0be"} Jul 09 07:41:15.570893 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:15.570493 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s54zf/must-gather-sxzr6" event={"ID":"fdeaa671-9da7-4e28-93de-819cf4498a79","Type":"ContainerStarted","Data":"c8aec93dac69895812a809efe48255ad1cebdb475f7619c5c60b6ed78628f2a7"} Jul 09 07:41:16.711716 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:16.711688 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4w92n_19428e47-7c22-4917-a531-e27501b88251/global-pull-secret-syncer/0.log" Jul 09 07:41:16.816537 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:16.816497 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f7mth_6ffe5743-2bfd-4475-b676-351e1d6fefbe/konnectivity-agent/0.log" Jul 09 07:41:16.960453 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:16.956595 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-42.ec2.internal_afa10e102bff134ec3a47ae4ffe0aeb1/haproxy/0.log" Jul 09 07:41:21.261362 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.261329 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/extract/0.log" Jul 09 07:41:21.287828 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.287786 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/util/0.log" Jul 09 07:41:21.314038 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.314008 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pz45_ddb5e708-5449-4763-949a-d1da3d934776/pull/0.log" Jul 09 07:41:21.364806 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.364765 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/extract/0.log" Jul 09 07:41:21.417029 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.416999 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/util/0.log" Jul 09 07:41:21.435957 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.435923 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlzbj_1853565c-d4f4-44cb-b698-5b0a4c1dd611/pull/0.log" Jul 09 07:41:21.464112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.464080 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/extract/0.log" Jul 09 07:41:21.492670 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.492638 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/util/0.log" Jul 09 07:41:21.518311 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.518229 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73h7tvm_7be316c7-df7b-478b-979e-6200b7ca5255/pull/0.log" Jul 09 07:41:21.547263 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.547231 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/extract/0.log" Jul 09 07:41:21.570957 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.570919 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/util/0.log" Jul 09 07:41:21.598371 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:21.598342 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b26k2_30c0e91a-a16a-49bd-a8e3-40e7b3baf599/pull/0.log" Jul 09 07:41:22.087410 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:22.086560 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69d759fc69-rq7d9_a07d1665-71c0-48a3-9de2-bcc8be4519bb/authorino/0.log" Jul 09 07:41:22.119344 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:22.119229 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7mvq4_5675ae79-b653-4e49-80ce-c63881408de4/manager/0.log" Jul 09 07:41:22.159024 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:22.158993 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nz2hn_13b6ef49-736f-4d25-821a-2d547bbb4047/manager/0.log" Jul 09 07:41:22.239349 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:22.238148 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dkkxx_1243bfae-0fd7-4ee7-8450-b48a0aacc0cf/registry-server/0.log" Jul 09 07:41:22.437040 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:22.436996 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cdcc7_abf20ae1-4d09-44cf-a368-e4b4079a8499/manager/0.log" Jul 09 07:41:24.658875 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:24.658830 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5df8f7c654-lb5kq_7fc9cc6f-fe90-47c9-bab3-379d15d059f3/metrics-server/0.log" Jul 09 07:41:24.730527 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:24.730479 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wpmm_e8f383b6-8045-45f4-a2a3-a50e0674607c/node-exporter/0.log" Jul 09 07:41:24.761441 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:24.761411 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wpmm_e8f383b6-8045-45f4-a2a3-a50e0674607c/kube-rbac-proxy/0.log" Jul 09 07:41:24.785141 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:24.785111 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9wpmm_e8f383b6-8045-45f4-a2a3-a50e0674607c/init-textfile/0.log" Jul 09 07:41:25.262429 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:25.262379 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kk5mf_3fdfd242-9d27-494b-b2e6-719951e8ad53/prometheus-operator/0.log" Jul 09 07:41:25.291832 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:25.291803 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kk5mf_3fdfd242-9d27-494b-b2e6-719951e8ad53/kube-rbac-proxy/0.log" Jul 09 07:41:25.318992 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:25.318956 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-h58qb_5963dd1d-5bed-4dc7-8249-a8c3a08e1320/prometheus-operator-admission-webhook/0.log" Jul 09 07:41:26.881120 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.881051 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s54zf/must-gather-sxzr6" podStartSLOduration=12.530195434 podStartE2EDuration="13.881026357s" totalImagesPullingTime="1.350830923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 07:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:41:15.587490525 +0000 UTC m=+2330.261347371" watchObservedRunningTime="2026-07-09 07:41:26.881026357 +0000 UTC m=+2341.554883205" Jul 09 07:41:26.885281 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.885243 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6"] Jul 09 07:41:26.894491 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.894465 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:26.896132 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.896104 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6"] Jul 09 07:41:26.994722 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.994675 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-lib-modules\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:26.994926 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.994755 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-proc\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:26.994926 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.994811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-podres\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:26.994926 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.994857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-sys\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:26.994926 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:26.994897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc8nd\" (UniqueName: \"kubernetes.io/projected/06dc70f4-2682-4d7c-9cbc-7931546c7519-kube-api-access-tc8nd\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.095478 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.095437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-sys\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.095726 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.095704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc8nd\" (UniqueName: \"kubernetes.io/projected/06dc70f4-2682-4d7c-9cbc-7931546c7519-kube-api-access-tc8nd\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.095974 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.095952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-lib-modules\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.096112 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.096097 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-proc\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.096246 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.096232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-podres\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.096535 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.096512 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-podres\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.097156 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.096733 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-lib-modules\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.097280 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.096774 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-proc\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.097371 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.095653 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06dc70f4-2682-4d7c-9cbc-7931546c7519-sys\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.106187 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.106161 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc8nd\" (UniqueName: \"kubernetes.io/projected/06dc70f4-2682-4d7c-9cbc-7931546c7519-kube-api-access-tc8nd\") pod \"perf-node-gather-daemonset-wc9b6\" (UID: \"06dc70f4-2682-4d7c-9cbc-7931546c7519\") " pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.209951 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.209918 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.380162 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.379195 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6"] Jul 09 07:41:27.678031 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.677988 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" event={"ID":"06dc70f4-2682-4d7c-9cbc-7931546c7519","Type":"ContainerStarted","Data":"e4021de30316d710e419704a6d511c409b94256e6c9105221c7ef2e851b6d38f"} Jul 09 07:41:27.678231 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.678059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" event={"ID":"06dc70f4-2682-4d7c-9cbc-7931546c7519","Type":"ContainerStarted","Data":"c9095dc70c99c361890482b3a8233439f50d15806ff49d9ba5407ba37374b5ba"} Jul 09 07:41:27.678316 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.678278 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:27.695784 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.695732 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" podStartSLOduration=1.695715833 podStartE2EDuration="1.695715833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 07:41:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 07:41:27.695070267 +0000 UTC m=+2342.368927114" watchObservedRunningTime="2026-07-09 07:41:27.695715833 +0000 UTC m=+2342.369572679" Jul 09 07:41:27.824250 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:27.824164 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c46fd74f7-szqcf_004bbad5-f5e3-4067-b706-db4948c5faf4/console/0.log" Jul 09 07:41:29.566224 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:29.566185 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pv68j_e2479eae-855f-4c16-b0e8-23b6bff78e0f/dns/0.log" Jul 09 07:41:29.588284 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:29.588242 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pv68j_e2479eae-855f-4c16-b0e8-23b6bff78e0f/kube-rbac-proxy/0.log" Jul 09 07:41:29.711684 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:29.711661 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztdwp_d977d649-07ee-4d4b-b64e-3f89c121db76/dns-node-resolver/0.log" Jul 09 07:41:30.253222 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:30.253188 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d67c8b49f-grbrn_0e28daf2-4c8b-463f-862a-9c72ee91a807/registry/0.log" Jul 09 07:41:30.280445 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:30.280415 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dnh77_0b07514b-c2af-4d17-99a6-37e4102ac780/node-ca/0.log" Jul 09 07:41:31.315079 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:31.315049 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf457qpr_36885451-6534-4ed7-9ebe-c95229740495/istio-proxy/0.log" Jul 09 07:41:31.568376 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:31.568292 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/0.log" Jul 09 07:41:31.630358 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:31.630324 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/1.log" Jul 09 07:41:31.765119 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:31.765083 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7sr8x_1d19221d-a578-4069-b83a-0ea24f018d90/payload-processing/0.log" Jul 09 07:41:32.495710 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:32.495679 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g4vhc_f99831a5-a724-478f-8146-181ea93651c5/serve-healthcheck-canary/0.log" Jul 09 07:41:33.021076 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:33.021046 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jd546_3e3b2de8-ae2b-4aaa-949a-fed1854767f2/kube-rbac-proxy/0.log" Jul 09 07:41:33.046840 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:33.046810 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jd546_3e3b2de8-ae2b-4aaa-949a-fed1854767f2/exporter/0.log" Jul 09 07:41:33.070046 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:33.070011 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jd546_3e3b2de8-ae2b-4aaa-949a-fed1854767f2/extractor/0.log" Jul 09 07:41:33.699106 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:33.699065 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s54zf/perf-node-gather-daemonset-wc9b6" Jul 09 07:41:35.970624 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:35.970545 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65ccf64d86-6tfjx_a4756ffb-aebe-4995-99fc-b41c59a5ba73/manager/0.log" Jul 09 07:41:36.106621 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:36.106592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c58b47-bz7pz_d22468e8-ccc7-4d4a-a46e-a77d4186320e/manager/0.log" Jul 09 07:41:36.127121 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:36.127084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c58b47-bz7pz_d22468e8-ccc7-4d4a-a46e-a77d4186320e/copy-manifests/0.log" Jul 09 07:41:37.537473 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:37.537366 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf457qpr_36885451-6534-4ed7-9ebe-c95229740495/istio-proxy/0.log" Jul 09 07:41:37.609273 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:37.609236 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bbcbb4454-2n942_9b77e34b-ea45-49a3-9d37-81a27b72bc38/manager/0.log" Jul 09 07:41:37.762845 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:37.762815 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/0.log" Jul 09 07:41:37.818771 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:37.818691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/1.log" Jul 09 07:41:37.937095 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:37.935039 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7sr8x_1d19221d-a578-4069-b83a-0ea24f018d90/payload-processing/0.log" Jul 09 07:41:39.078974 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:39.078942 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf457qpr_36885451-6534-4ed7-9ebe-c95229740495/istio-proxy/0.log" Jul 09 07:41:39.325913 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:39.325861 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/0.log" Jul 09 07:41:39.384527 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:39.384499 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mfbgr_6d1f9a59-576b-40f6-bd3a-98fcf6249a1a/istio-proxy/1.log" Jul 09 07:41:39.508815 ip-10-0-143-42 kubenswrapper[2573]: I0709 07:41:39.508777 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7sr8x_1d19221d-a578-4069-b83a-0ea24f018d90/payload-processing/0.log"